Most days, it feels like I’ve been working in the lean startup movement for 20 years. The tech startup industry has been through so many massive changes since we first got started, we’ve packed many years of experimental cycles into such a short time. Do you remember what year “The Lean Startup” by Eric Ries was first published?

However long it’s been, how do we know we’re doing a great job? Before you chuck in your full-time job and begin the startup founder journey, how do you know for sure an accelerator or incubator is going to improve your chances of success?

Since incubators and accelerators are such a recent phenomenon and teaching/practicing methodologies that are still changing in response to a rapidly-changing industry, I don’t think anybody can yet say with authority what constitutes good practice in the space.

If a poor accelerator program manages to secure an intake of world-class startups that go bananas almost despite the accelerator program, how do we distinguish that from a great accelerator program that makes a significant difference for a cohort of middle-rank startups?

A Second Century Ventures study published recently in VentureBeat found that in the US, it was necessary to observe the outcomes of a graduating cohort for four years after graduation from an accelerator before it was possible to draw a conclusion about performance. Only problem is: we don’t have that much data on the Australian accelerators yet.

Startmate is Australia’s earliest accelerator and its first cohort graduated in 2011. AngelCube started in 2011 but didn’t graduate a class until 2012. Pollenizer was founded in 2007 but was more akin to a digital agency with an equity stake until 2011-ish. BlueChilli began offering an accelerator program in late 2012.

I think the best way to improve the performance and ensure the value of incubator and accelerator programs is to encourage transparency and validate and publish data for key metrics. One key metric for accelerators could be the delta in the total capitalisation of the accelerator’s graduating startups over five years. Another might be the return to investors in the accelerator.

Some have called out the “11 millionaires created” stat on the new BlueChilli homepage as a ‘vanity metric’ and asked if really, those startup founders are just ‘paper millionaires’. Well, of course they’re still paper millionaires — the earliest of them has only been operating their startup for two years. (Some would say being “just” a paper millionaire is a whole lot better than not being a millionaire at all.)

Some have called for industry or government regulation and accreditation, but surely any accreditation or regulation regime would lag the current industry best practice by at least a year and more likely a parliamentary term.

And accreditation would forever be rewarding dogmatic adherence to outmoded methodology. Accelerators learning at the coal face about what’s working would be unable to practice new methodology without risking their accreditation.

I want Australia’s startup accelerators to change, adapt, learn and grow as rapidly as possible. As an investor, startup co-founder and startup advisor, I don’t want our startup education system subject to the operating principles of the industrial era. The reason it feels like we’ve been at this for decades is because our industry learns and grows faster than any other. Rapid learning through early exposure to the marketplace is what defines us.

If Second Century Ventures is correct, we’ll begin to see how well Australia’s startup accelerators have been performing in a year or two. In lean startup terms, an eternity, I know. I’ll be waiting impatiently too…

By the way, “The Lean Startup” by Eric Ries was first published in 2011!

Filed under:   accelerator   Australia   industry