14 years ago when I was a young bright-eyed Software Engineering student I remember sitting through endless classes in which wise professors would repeatedly warn us of two fundamental Engineering rules:
Never start building anything before you have designed it and Never release anything until it has been thoroughly tested.
Of course thanks to our youthful ignorance and passion for coding my fellow classmates and I regularly ignored these wise words and as a result we delivered many failed projects and fancy-looking systems which were either unusable irrelevant or complete failures when tested. We chose to learn the hard way.
Thankfully today’s ITS Engineers know better.
In fact these days they don’t just test products they hold week-long testing festivals known as ‘Testfests’. These extraordinary events bring commercial engineers designers and testers together to assess the functionality of products in an almost real-world situation before the products are released. This is clearly good news for an industry built around technologies that need to communicate reliably.
So this month in an effort to understand these curious events and uncover their high-tech secrets I have been on a mini Testfest quest (perhaps called a Testquest?).
Here is what I discovered:
They really are festivals
Like other festivals Testfests are community events which are held around a specific topic usually an ITS standard and they occur during a large meeting in a single location. However unlike other festivals ITS Testfests are practical events. Generally speaking they do not involve dancing and eating (although both are allowed) and they focus almost exclusively on testing and validation. They are sensible gatherings motivated by solid engineering principles such as reliability standardisation and interoperability.
They are organised by engineers for engineers
Testfests are at their core engineering events. They are organised and designed by leading industry experts often those who are involved in defining industry standards. In fact the events are often organised in collaboration with standardisation organisations such as TISA and ETSI.
They are not certification sessions
Testfests validate products to ensure that they work when connected to and communicating with products made by other manufacturers. By connecting systems together in the same room engineers can see if the technology works as they had expected. If they do not work the test data should help them understand why not and what to do.
Usually this involves testing a specific implementation of a technological standard but as mentioned before – the focus is on interoperability. Does product X work with product Y?
ITS standards like other standards may have inconsistencies or ambiguities which manifest themselves in ever-so-slightly different implementations. If this happens then ‘testfesting’ is a great way to uncover potential conflicts arising from these inconsistencies; however this is not the same as testing your product against the standard.
Confidentiality and cordiality rule
When I interviewed Sébastien Mure Project Manager at ERTICO about Testfests he was not just happy but genuinely enthusiastic about the atmosphere of openness and cooperation he has witnessed at previous Testfests. Understanding that many of attendees are commercial rivals Sébastien stressed the importance of strict adherence to non-disclosure agreements and the climate of trust that this builds. Testfests are after all about improving the interoperability and quality of systems not beating your competitors. He believes that the energy and excitement on display during these events comes from the desire to make things work and to make things work well.
I have to admit his enthusiasm was infectious and while I was smart enough to quit Software Engineering before I did any serious damage to the reputation of the industry – I am very much looking forward to the upcoming ‘fests’ which are being planned for later this year. The first a TPEG Testfest will take place in May (see www.tisa.org for further information) and the second an eCall Testfest will take place during the summer. See www.heero-pilot.eu.
Both events look set to be a huge success and if they are I think we will see many more Testfest events in the coming years.
By Ian Bearder
Communications Officer
ERTICO – ITS Europe.
i.bearder@mail.ertico.com
How a Testfest works
1. The event is organised advertised and booked for specific dates. Attendees sign confidentiality and non-disclosure agreements.
2. Industry experts define the test scenarios and specifications which will be used
3. Attendees arrive on day one and setup/calibrate their equipment
4. Testing begins. Tests are carried out using a ‘Championship’ formula which means each company or group will test their product against all others.
5. Each test is carried out in accordance to specific rules and each is marked as OK Not OK Not done or Not applicable.
6. Depending on the number of tests and the number of participants each day allows for about 3-4 testing sessions.
7. The day ends with a debriefing and discussion in which common problems can be discussed together with industry and standard experts.
8. If they wish participants attend evening social events. Others retire to their hotels to continue work on their products and services.
9. At the end of the festival attendees receive the results of their own tests and may participate in a final discussion session and debriefing.
10. Attendees leave and return to their work and hopefully use their experiences to enhance their products for the benefit of the industry and their end users.
Link to original Article
http://www.ertico.com/my-testfest-quest/
Original Publication Date: Tue 19 Mar 2013
