It all began…

It is in September of 2010 that the idea of doing a programming contest came to life. Coveo was a startup with highly passionate employees, ready to grow and bring more passionate people on board.

Did I say passionate?

Our primary source for hiring talented and passionate - I know, but it’s really important to us - software developers was (and still is) universities. We bring students on board during an internship and show them what it’s like to work with us - the challenges, the environment, the people, … - it usually works really well and we end up hiring almost 33% of the interns.

Back in 2010, Coveo was little known in Quebec City, let alone Montreal or Sherbrooke. We had to find a way to spread the word on how Coveo is such a great place to work at.

I was at the coffee machine when Jean (our CFO) came in and asked:

Jean: How’s hiring doing?

Marc: It’s been better. Our last internship posting had only got 4 applicants and we will be lucky to get 1 intern.

Jean: What can we do to get more applicants, to get our name out there?

Marc: *joking* We should do a programming contest for students!

Jean: What a great idea - let’s do that!

Ok, that was not totally a joke, but not totally serious either! The idea came from a programming contest held in Quebec City called “Le Bivouac Urbain”, where contestants had to create a full video game in 48 hours.

Defining the contest

The decision to do a programming contest was made - yeah, that’s really all it took to start the project! But we still needed to do a lot of things to make it a reality and the most important one was defining the challenge.

But first, let’s talk about the name, because this contest needed a name. Since I’m not that good to come up with names, I asked all the people from our R&D team (27 people back then) for suggestions. Here’s a few I received:

• Programmathon Coveo

• Coveo 8 hours of Search

• Coveo Search Blitz

• CoveoPalooza

• Coveo Challenge

No surprise here, we decided on Coveo Blitz.

Now, back to the challenge itself. At first, we floated the idea of doing a 24 hour contest where students had to create the best web site they could based on the criteria we provided. We brainstormed around that idea to finally change it completely and tie it a lot more to our business.

We wanted people to know that creating a search engine that works is challenging. Here’s what was going to be the first project for Coveo Blitz:

• Create a search engine to index 45,000 documents (48,223 documents to be exact)

• Support boolean operators and exact phrases

• Rank documents by relevance

• Create a web page to display the results

The final score would be based on several criteria: performance, code quality, quality & design of the UI, and of course, results precision. The correction would be manual, meaning that Coveo’s staff would evaluate the different aspects of the project.

Preparing the day

Our objective was to get 5 or 6 teams of 4 students. Based on the limited success we had attracting students, achieving that goal would definitely help spreading the word about Coveo. We reached to the different universities & colleges to promote the contest. We received more than 12 registrations in a few weeks! We were really excited, but at the same time, …

That was a problem, since we initially wanted to host the contest in our office, but we didn’t have a room large enough for all these people + staff! We fixed this through a fantastic collaboration with Laval University that offered to rent us a classroom. We decided to hold the contest there. We also limited the contest to 8 teams and asked all team to provide a small demo project to classify. I evaluated all those projects and selected the 8 teams.

One of the teams presented an algorithm to compute stuff I can’t remember, but I sure remember that they put a nice image while the computation was running.

Image presented while the algorithm was computing

We wanted the contest to be one of the best experiences the students would have in such an event so that they would have a highly positive opinion of Coveo and its people. First thing we did was to take care of feeding the participants, partly dressing them and teasing them with the prize for the winning team. Here’s a list of things we provided:

• Free drinks (juice, water, soda and of course coffee and Red Bull)

• Free food (breakfast, lunch, dinner, snacks)

• Free beer - yeah, for real!

• 4 Alienware laptops for the winners

• T-Shirts with nice quotes - that’s also kind of a publicity for us afterwards as shown in this tweet

One participant tweeting about his Coveo Blitz T-Shirt

Coveo Blitz 2011 T-Shirt (Black for students we hired, blue for staff & white for students)

Since the contest was based on performance, we wanted to make sure that everyone had the same chance at winning the contest. We decided to provide laptops for all contestants - that meant 32 laptops to buy, configure, move, install, move back and sell! I know, I know! At the time, it seemed like a good idea, but in hindsight, that was a bad idea and definitely to put in the don’ts list!

We spent a number of hours preparing the room the day before the contest and cleaning it after the contest. We brought a router and quite a few number of network cables & electric extensions.

A lot of people at Coveo got excited by Coveo Blitz and helped make it a reality. I can’t stress enough that without their help, Coveo Blitz would not be possible. And that is true, year after year, so THANK YOU!

The router and cables

The room and participants

Lessons learned

We learned several things by doing all these editions of Coveo Blitz. Here are the most important ones, at least, for me!

You are never too prepared

We are now at our fifth edition of Coveo Blitz and I can tell you that after every edition, we told ourselves that the following year, we’ll be more prepared. We start working on the next edition several months in advance, often 8-10 months before, but you need more than enough time to succeed. Because as says the Parkinson’s law: work expands so as to fill the time available for its completion.

I suggest to take the preparation of a contest like any other project and set milestones. Make sure you respect those, otherwise, you may end up like this on the day of the contest!

Coveo’s version of Grumpy Cat

Favor automatic correction over manual

We planned about 60 minutes to correct the team solutions in the first edition and it took more than 3 hours! Also, automatic correction opens a lot of opportunities - see below.

Multiple evaluation runs with dynamic scoring/dashboard

If you ask anyone who participated to Coveo Blitz, starting in 2013, what was the highlights, I bet they will tell you “dashboard & scoring”.

Being able to do multiple evaluation runs with a live dashboard displaying real-time score as the run is executed is just awesome! It’s like watching a sport game, everyone is standing and cheering!

An evaluation run at Coveo Blitz 2013

Host the event in your office

If possible, hosting the event in your office is a plus. At least, it was for us.

• It shows the candidates your work environment and gives them a feel of what it might be like to work there - although we’re not always working on 8 hours deadline!

• It is easier to setup since all the infrastructure is there - network, desk, chairs, …

• You can prepare in advance since it’s in your office

Have people bring their own hardware

I said it earlier, do not supply the computers for the people. Have them bring their own laptops, desktops or mainframe, whatever they want. When you do the evaluation, use a method that is not based on their hardware to compute the score. For example, we now use Amazon Web Services to run the contest and do evaluation runs, which we can somewhat control (by machine type, for example).

Innovate

We tried to innovate at every edition of Coveo Blitz by modifying the challenges but also by modifying the infrastructure - dashboards or achievements, for example.

We also have a few surprises for the 2015 edition and it started when we launched the web site. We added a small challenge to register to the contest. Students had to create a small REST API that replied a proper answer along with all the information about the people in their team. We had fun creating the error messages for wrong response, had you noticed them? :)

One of the teams built a server on a Nintendo DS - How cool is that?!

Free food & beer

I think that giving people free food & drink during the contest is a plus. We also have free beer to celebrate at the end of the day, and a tower of pizzas. Everyone appreciates this and I bet the Coveo team appreciates the beer even more than the students: celebrating the success of the contest after having worked on this single day for the last several months is a must!

This is also the moment where everyone can talk together and where we make contact with the students. We can talk about what the work is like at Coveo and also talk about current opportunities. Remember that the #1 reason for the contest is recruiting.

The pizzas at Coveo Blitz 2013!

Fun facts of the 2012 edition

• 48 participants

• +30 Coveo employees worked on this edition

• +600 hours invested by Coveo employees

• 166 tweets related to this edition

• 64 t-shirts

• 35 tables & + 60 chairs

• 657 meters of network cable

• 171 meters of electric extension

• 16 network switches

• 15 power bar & 3 UPS

• 1 wifi network

• 60 laptops & 2 desktop

• 1 daskeyboard ultimate

• 1 whiteboard

• 4,900 lines of code for the Coveo platform

• A 7GB database containing 10,715 products

• Around 200 evaluation runs

• 10 x 12 donuts

• 6 x 12 muffins

• 50 litres of coffee

• 60 litres of juice, water & soda + 7 litres of Red Bull

• 100+ sandwiches

• 200 slices of pizza

• 407 pictures

• 2 hours of video

In conclusion

The first edition was a success and we got a few people hired out of that edition. We also got the word out, if not about how cool Coveo was, at least how free the beer was at the contest! :)

We hired more than 35 people in R&D since the first edition of Coveo Blitz. Out of those 35, 13 participated in Coveo Blitz and 32 learned about Coveo from Coveo Blitz. Another impact is the number of interns we get in R&D every year. In the summer of 2013 we welcomed 19 interns and 15 in the summer 2014. All of them participated or heard from friends about Coveo Blitz.

If you know me, you know I really like pie charts, so let’s try to represent these numbers in a nice pie chart!

When I see that most people I have in interviews know about Coveo Blitz or probably heard about Coveo from Blitz, I tell myself: “Mission accomplished!”, and, I’m always happy to see Coveo Blitz mentioned in someone’s resume contest section.

In just a few weeks, it’s going to be the fifth edition of Coveo Blitz. The Coveo team can’t wait to live that. Several people have been working on the project since last March - yes, it takes time to build the challenges, test them and create all these nice dashboards!

I hope to see you then!

Logos from all Coveo Blitz edition to date