Contributing to future road safety with annotations on the edge

The underlying need for automatic annotation on the edge is simple: Zenseact wants to extract value from the data collected by all cars where the company's technology is used.

Developing technology for autonomous cars means crunching a lot of sensor data. Zenseact's platform uses cameras, LIDAR, high-precision GPS, and other sources to understand what is happening in the surroundings and have the car act accordingly.

“We have our own fleet of development cars, from which we collect data to train our models. But compared to the number of cars where our customers have deployed our technology, our fleet is much smaller”, says Benny Nilsson, Head of Emerging Technologies at Zenseact.

Therefore, Zenseact would like to access the data collected by the sold cars to train their models. However, this is so far an unsolved problem. Why? GDPR and privacy regulations is one challenge. 

“This comes down to both the privacy of the driver, if he or she drives to sensitive destinations, and also the privacy of pedestrians and other persons in the car's surroundings.”

Another challenge is the vast volume of data collected. Transferring all data from the cars to central servers would require a lot of bandwidth. All in all, this means that the sold cars are an untapped resource of data for Zenseact.

"We need a way to annotate data at the edge"

Instead of moving the raw data from the individual nodes to a central data storage for training, edge learning and decentralized AI allows for training the models at each node and sending the result back to the central location where the information contributes to a larger model. This way, no sensitive data is transferred and the data volume drastically decreases.

“The majority of Zenseact autonomous driving tasks require supervision in the form of annotations for training at the edge. Traditionally, highly skilled human annotators provide these annotations that are often not accessible at the edge. Even if we could find a way to incentivize the customer cars’ occupants, there is no guarantee they would follow the detailed annotation guidelines properly. Therefore, Zenseact needs new methods to generate high-quality annotations automatically at the edge”, says Mina Alibeigi, AI Researcher in Zenseact’s Technology and Collaboration Team.

"Running a hackathon with external participants is an excellent way to get inspiration from other sectors"

Zenseact therefore teamed up with AI Sweden to challenge the AI community to find a solution. By inviting external teams to the challenge, there was an input of a variety of ideas from developers with different background and working sectors other than the automotive industry. 

“Automating annotations the way we need to and on the edge is still an open question. We have run internal hackathons before, but those have had a much more defined solution space. The challenge in this hackathon is on a more conceptual level, and because of that we decided to invite external teams”, says Benny Nilsson.

The Challenge resulted in five different ideas on how the annotation problem could be solved - providing Zenseact with concrete solutions and ways forward.

But there is another value in a hackathon with external teams:
“It becomes somewhat of a recruitment process, both for finding individuals to hire, or researchers or companies to work with”, says Benny Nilsson.

Additionally, running the Challenge together with AI Sweden in the Data Factory gave the Challenge a neutral setting. 

“Intellectual properties are owned by the participating team. But uploading their code to Zenseact's servers could still have been a show-stopper for some. The neutral setting of the Data Factory made a big difference. Another benefit from Zenseact's partnership with AI Sweden was AI Sweden's network, which was used to market the hackathon” says Benny Nilsson.

 

Facts
Read more on the EdgeAnnotationZ Challenge organized by Zenseact, AI Sweden, RISE, Lund University, Chalmers Industriteknik and Univrses, in collaboration with CGit and MobilityXLab.
Learn about the winning entry by Annotell.
Curious about the dataset that was used? Check it out and learn more.
Want to donate a dataset for the AI Sweden network to work on? Contact Ebba Josefson Lindqvist.
Interested in setting up a hackathon challenge? Get in touch with your partner manager at AI Sweden.