Forem: markrossatos

AWS Deep Racer Event Hosting — In-Person Racing

markrossatos — Mon, 26 Sep 2022 12:45:48 +0000

This is the third and final part in my three part series on AWS Deep Racer. If you’ve arrived here directly the first part in the series can be viewed here, and mainly covers setting up AWS Deep Racer in a single AWS account using multi-user mode. The second part can be viewed here, and mainly covers running a virtual online race. I’d recommend you read those first, then return to read about the physical racing. There’s quite a lot to consider when hosting an in person race so let’s get started…

The room you host the event in needs to be a good size. The re:Invent 2018 track we had printed came in at 7.9m x 5.2m, and some of the newer tracks are even longer! I would strongly recommend printing it at the correct dimensions to match what has trained in the virtual world to give your models the best chance. You need to consider what surface you’re laying the track on, and how you’d keep it stretched out. We’ve laid our track on a tiled floor previously and it’s been great, this time we laid it on a carpet-tiled floor and because it had a little bit of give in it we did experience a few wrinkles, which increased with use as people were following the car around. There are other options too, for example you could create your track with white tape on an existing dark coloured floor to keep your costs to a minimum, but a printed track certainly looks more professional. If you need help deciding what to build and how then AWS provide a nice guide here.

Lighting and what objects are in the room is another important consideration. Trying to remove or cover white objects that the car may be able to see is a sensible idea, otherwise you may find your models behave differently than they did in the virtual world. We had to cover some white support posts as well as turning off some screens with a white background that seemed to be interfering with the cars. Another technique, which has the added benefit of stopping the cars getting too far away when they veer off track, is to build barriers. Our barriers were knee high, which logistically is a lot easier than the waist high ones you see at AWS re:Invent and the summits, however I did notice when watching the video of what the car camera was picking up it could see over the barrier at certain points. Try and make the lighting as consistent as possible too, to avoid lots of shadows, or particular bright spots versus the rest of the set-up, we used our standard room interior lighting and for consistency pulled down the blinds as half of the walls in the room were floor to ceiling glass.

Moving on to the cars I would strongly recommend you have multiples of everything available, after all ‘everything fails all the time’ as Werner Vogels would say! Having multiple cars gives resilience in case of failure, so you can carry on racing whilst troubleshooting any issues. It also allows you to get a production line going, so whilst one car is on the track racing the next car can have a model loaded onto it to avoid lots of time when there aren’t cars racing on the track and keep the audience engaged. Extra batteries on top of what are inside each car would also be a sensible addition, so batteries can be charging whilst cars are racing. Details on batteries and other spare parts can be found here. Details on how to set-up and calibrate the cars can be found on the AWS website, you’ll want to calibrate the cars initially and then if you start to have poor model performance during the day a recalibration may be required as heavy crashes into the barriers could impact calibration.

A couple of other useful additions to your set-up will be a mechanism for timing each lap and a wireless network. These could range from a simple manual timer (e.g. phone / stopwatch), to an automated timing mechanism on the start / finish line using a pressure sensor and a Raspberry Pi for a more professional job. A separate wireless router is useful too, as you can preconfigure everything to that network so you don’t have to reconfigure things if you take them to different offices, and all the higher bandwidth activity can all stay local.

Once you have everything in place I strongly recommend you test everything. This allows you to test your team that are running the event, so everyone know what they’re doing, and it also allows you to test the environment, in case anything is adversely affecting the cars.

Our in-person race was our top 30 participants from a virtual qualifying round. We gave them additional hours of training to further alter and improve their models from the end of qualifying to the day of the race. We had around 20 people from the UK qualify who could get to our London offices, whilst the other 10 qualifiers were from further afield (USA, Guatemala, Denmark, France, Romania and Germany). To keep our remote colleagues engaged we set-up a couple of webcams to capture the action from different angles, and with the help of some colleagues streamed the webcams and a live leader board into the metaverse, complete with a representation of the track!

We gave each racer an initial time slot to come and test out their models. some models transferred well to the physical world, whereas others that had performed well in the virtual world struggled a bit. In machine learning this problem is known as being ‘over fit’, some of the models that did very well in the virtual world had become overly reliant on the training data (the virtual environment) and weren’t so good at generalising and overcoming things they’d not previously seen (e.g. a wrinkle in the track, a shadow etc.). This was a good learning point for participants and something that does transfer into the real world, a road isn’t a sterile environment it could have rubbish on it, a pot hole or markings that have faded. Once everyone had been given a round we then invited back the top 10 for a further shoot out to see if their times could be improved.

The results of the top racers were impressive, and interestingly the podium for the in person event was completely different to the virtual qualifiers, with each of the top 3 going quicker in person than they’d managed in the virtual qualifiers.

Atos and Cloudreach in-person leaderboard top 12

Our winner, Marco, had an excellent model. Not only was it the fastest but it consistently went around the track, in his first run he posted a sub 9s time and then in the top 10 shootout he managed to lower that time to an incredible 7.939s.

Fastest lap from our live finals

A worthy winner and in the process, to everyone’s surprise, he bagged himself an all expenses paid trip to re:Invent presented to him by one of our Atos OneCloud leadership team, Santi Ribas!

Presenter Santi Ribas (left) with our podium left to right (Nickson, Marco, Simon)

Participants and AWS colleagues who helped to run the event.

Finally I’d like to thank those that helped to organise and run the event. My Atos colleagues Matt Knight and Neil Clark who helped with the virtual and physical races, and my colleagues Vrushali Malankar and Kshitij Bhatnagar for creating the metaverse. I’d also like to thank my AWS Colleagues Sathya Paduchuri, Bharath Sridharan, Rajan Patel, Stuart Lupton, Pete Moles and Jenny Vega.

AWS Deep Racer Event Hosting — Virtual Racing

markrossatos — Tue, 20 Sep 2022 13:43:02 +0000

Details of our recent virtual race

This is the second part in my three part series on AWS Deep Racer. If you’ve arrived here directly the first part in the series can be viewed here, and mainly covers setting up AWS Deep Racer in a single AWS account using multi-user mode. I’d recommend you read that first, then return to read about the virtual racing.

Once your participants have access to AWS Deep Racer you’re going to want to create a race for them to enter to ramp up the gamification side. To host a virtual race you’ll need someone with the administrator privileges to go into the Deep Racer Console and navigate to ‘Racing League’ and then ‘Community Races’ on the left hand side. You should then have the option to ‘Create Race’ on the right hand side, this will then present you with various options for your virtual race format.

Options for a virtual race

Out of the options available I would recommend thinking about whether this is part of a series, is going to include an in person race, and the level of previous experience your racers have with Deep Racer. For example our virtual race was designed for the majority of participants who’d never used Deep Racer before, and was a ‘qualifying round’ for an in person finals. We chose to run a ‘Classic race’ because people could set times and then refine models to set their fastest possible lap, giving them a chance of having some really good models for the in person event. If your race was a one off with no follow up you might prefer a ‘Live Race’, allowing you to arrange a video conference and all watching the race together in a ‘shoot out’. We chose ‘time trial’ as the simplest race for new competitors, and because it aligned with our plans for the in person race, but if you’re running an event for people with significant experience you might want to choose an alternative. You can then choose the time period your virtual event runs, although it’s worth noting people can’t pre-join the race ahead of opening time, they’re only allowed to join once the race time period is open.

Further options for a virtual race

Choose your track, which if you’re planning to run a follow up in person event should be the same track as will be used there, and then choose how you’ll rank times, number of laps, and the penalty for going off track. Once you’ve created it you should have a hyperlink you can use to share with your participants to join your virtual race.

I would strongly recommend running an event kick-off. For newbies an introduction to Machine Learning and reinforcement learning is essential to help understand the concepts, and once they’re put in simple terms with some analogies the high level concepts of reinforcement learning are soon understood. Our participants had a great overview from AWS on Machine Learning in general, reinforcement learning, reward functions and hyperparameters within Deep Racer. As a dog owner I think this was my favourite slide.

AWS slide on reinforcement learning in the real world

AWS conceptual overview of model training

AWS Deep Racer model behaviour is influenced by a reward function and hyper-parameters.

The reward function is written in python, examples of which can be found here. Reward functions can take a number of input parameters from the car, they can be found here. They include parameters like the current speed of the vehicle, it’s position on the track, the amount of progress made around the track and more. These can be supplemented by importing python modules, for example time, math and others might be useful for more complex models. I’ve seen some extremely complicated models do poorly, and some extremely simple (in terms of number of lines of code) do extremely well, and vice versa! The important thing to note is you’re coding something to reward an outcome, rather than coding a precise action, this way the model is able to solve a complex problem like driving, without thousands of lines of code.

Simple reward function from AWS examples

More complex reward function importing math and calculating direction in relation to the track using waypoints and progress

Hyperparameters influence how the models train. This post isn’t supposed to teach you the inner workings of machine learning, and I’m not a PhD mathematician, so if you want to really understand the inner workings I’d recommend taking a course. An AWS focussed example would be the A Cloud Guru AWS Machine Learning Speciality course, which because the certification curriculum covers machine learning theory as well as AWS service implementation gives a great overview. Generally speaking you’re tuning these hyperparameters to either favour exploration (in our case explore the track, try and find a way to maximise the reward), or exploitation (refine and make incremental gains on a path around the track that is working). There are instructions in the ‘info’ section of the page where you configure them.

Sample of hyperparameter options and associated explanation in the AWS console

It’s important to communicate what car and track you want participants to train with, particularly if you want to follow up with an in person race. As the car is learning a track during the training it won’t perform well if you move it to another track for the race. Also it’d be no good training with a dual camera / lidar car, if your race car is single camera / no lidar.

It’s also important to communicate to people how to enter a model into your virtual race. Although you will have provided a hyperlink to participants as I described earlier which they can use to register against, they’ll still need to submit their individual models in to your race. Unfortunately at the present time the Deep Racer console doesn’t provide the most intuitive way to do that, in two different places! First when creating the model itself users have the opportunity to ‘Automatically Submit to the Deep Racer League’, this is the first place participants can submit their model to the wrong race, you’ll want them to deselect that option, otherwise they’re entering a public race most likely on a completely different track!

First place to submit to an AWS public race, rather than your private race!

The second place participants can enter the wrong race is after the model has trained. There’s a nice button entitled ‘Submit to virtual race’, unfortunately once again that takes the racer down a path of entering a public race most likely on a completely different track!

Second place to submit to an AWS public race, rather than your private race!

To ensure participants enter your race it is best to advise them, once they’ve registered via the hyperlink you provide, to navigate to Racing League -> Community races and then scroll down to find the correct race.

Entering your private virtual race

the screen should then show the event with a drop down list for them to select the appropriate model.

Enter race screen with drop down list for model selection

It’s important during a virtual race to think about how to communicate and engage with participants. As we’re Office 365 users I arranged a Microsoft Team with channels for announcements and Q&A, so participants had somewhere to go for reference information or recordings of calls we’d done like the kick-off event. I kept posting updates to keep up interest. Number of days to go, along with a summary of the day and a video of the current leading lap, which can be downloaded from the leader board screen (right click and save when the video is running) were my frequent posts. I arranged a number of drop in ‘surgeries’ at various points for the duration of our virtual race so participants could come along for help on getting started, troubleshooting issues or advice on how to tune their models. It was during these surgeries that the issues I’ve describe above like submitting a model to the public race came to light. In addition people had issues with permissions after about one hour, to do with session timeout. Logging out and back in via the ‘Your racer profile’ or completely out of the AWS Console usually did the trick, although one unlucky participant lost a complicated reward function they were authoring in the console at the time, so if you’re creating something complex it may be worth creating it outside the Deep Racer reward function screen and pasting it in.

Overall our virtual race was a success and has set us up nicely for the in person event. We had over 1,100 hours of training across 85 participants and we had great feedback that participants learnt something and had fun whilst doing it. Three people went sub 9 seconds, with 15 people going sub 10 seconds. Only 2.2 seconds separated our top 30 qualifiers who have been invited to our in person event. Our fastest qualifier came in at a rapid 8.394 seconds, video of their lap below!

Fastest lap from our virtual qualifying event

We now progress onto the physical race, and my write up on how to set that up and and how it went will conclude this Deep Racer trilogy of blog posts…

AWS Deep Racer Event Hosting — Setup

markrossatos — Wed, 14 Sep 2022 11:41:39 +0000

AWS Deep Racer cars on a physical track

I’m Mark Ross, Lead Cloud Architect at Atos, specialising in AWS. I’ve been an AWS Ambassador since 2021 and joined the AWS Community Builder program in 2022. I’ve set-up an AWS focussed community interested in training, certification and working with AWS technologies with our customers, our Atos AWS Coaching Hub. Our most recent event is an AWS Deep Racer competition, to allow colleagues to get hands on and learn some reinforcement learning in a fun and gamified way.

AWS Deep Racer provides the opportunity to train a reinforcement model in the AWS Cloud to race around a track. You develop a model based on a reward function, which encourages the car to undertake behaviour that gives it a greater reward, and discourages it from undertaking behaviour that will give it less / no reward. You’re not defining the output, but more the guiding principles the model should use to explore with. So reward for keeping on the track, going faster, not turning too sharply might all be useful strategies. I’m sure anyone with a pet or a child will relate I’m sure! The other major area to influence the model is the hyperparameters. Hyperparameters influence how quickly a model will train, which is in contention with how accurate the model may be, as there’s is often a trade-off between optimisation and exploration. Model optimisation is not dissimilar to how the ‘lean’ principles work, you might make incremental gains to optimise an existing process, but it’s not going to lead to an amazing new discover which might radically alter your ways of working or industry.

In order to host your own Deep Racer event you’re going to need to provide an environment for people to use. Fortunately capabilities for event hosting have been greatly improved with the release of AWS Deep Racer Multi-User Mode, which eliminates the need to issue individual AWS Accounts to people and all the headache that provides with set-up and close down afterwards, as even within an AWS Organization the limit on the % of accounts that can be closed in a month is a pain.

To make use of multi-user mode only a single AWS Account is required. Within that account each user requires the ability to access that account, with appropriate permissions. Options include an IAM user within that account, or if you’re operating this within an enterprise environment you could assume a role within the account, for example via using AWS Identity Centre (formerly known as Single Sign-On). Permissions required to use AWS Deep Racer are provided via an AWS managed policy (AWSDeepRacerDefaultMultiUserAccess), but if you’re using IAM users you’ll probably also want to add another AWS managed policy to allow users to manage their passwords (IAMUserChangePassword). As our event is a hybrid event with a round of virtual use, followed by a live in-person event, I also wanted to give people an easy way to provide their models to us ahead of time, to make it easier on the day to schedule everyone’s laps. To that end I created another policy that granted permissions to a shared S3 bucket I created, but only to a subfolder of the user’s IAM user name. The policy makes use of ${aws:username}, so there’s no need to create them per user.

S3 Bucket policy for users to be able to upload models — S3 bucket name has been obfuscated

To Enable multi-user mode you need to navigate to the AWS Deep Racer console in us-east-1 (the service only exists there at the time of writing), and on the right hand side there’s a ‘multi-user management’ section. You must be using an account with sufficient AWS permissions to set this up, an AWS managed policy can provide this (AWSDeepRacerAccountAdminAccess). If you open setup there’s an intuitive screen that will navigate you through enabling the feature within the AWS account, as well as sample email invitations you can share with participants. Once enabled you can also look in the ‘Monitor usage’ section to see how many training hours have been used and how much has been spent. This section is also where you can control the level of ‘sponsorship’ you provide to participants, in terms of how many hours of training they can undertake (currently costs $3.5/hour) and how many models they can store (currently $0.023 per GB-Month). One thing to note — you cannot set sponsorship levels until someone has registered, nor can you set your own customer default levels. Each participant starts with 5 hours and 3 models, and periodically you’ll want to go in and set to whatever you want to provide as annoyingly there’s no API for Deep Racer so you can’t do it automatically based on an event.

I decided it’d be sensible to automate the set-up as much as possible. The Deep Racer elements can’t be automated due to lack of API, but everything else can. I used a Microsoft Form (we’re Office365 users) to capture email addresses of interested participants. I’ve then created infrastructure as code and open sourced it in my GitHub repository here to create everything except the Deep Racer config automatically. I took the email addresses and used them as the IAM username (adding them to the variables file in my repo will allow you to deploy them). Running the code creates everything required permission wise and the S3 bucket for model storage, it then dumps out IAM usernames and passwords for you to issue to participants. I used the CSV output, converted it to an Excel table that could then ultimately be used as a data source for a mail merge (I was tempted to use AWS SES but my participants weren’t in domains my account had verified) to issue those details along with instructions for our event, along with some billing alerts so your event doesn’t get out of hand cost wise.

Once your participants have been issued with their credentials, or you’ve allowed them to assume a role into your account the user experience is pretty simple. They should log-in to the AWS account (or assume role if not using the code I’ve supplied) and navigate to the Deep Racer console in us-east-1. Because multi-user is enabled and they’re assigned the AWSDeepRacerDefaultMultiUserAccess policy they’ll see a prompt to sign-in to Deep Racer. Note — this is a different set of permissions to their IAM credentials, and are specific to Deep Racer (they can be taken elsewhere and used in other AWS Accounts later). These accounts can’t be pre-created due to aforementioned lack of API, so you need to get participants to create their own account via the link below the ‘Sign-in’ button. A quick registration process and code sent to their email address to verify should result in access being granted.

Once logged in participants will be able to use Deep Racer, however their permissions will limit their visibility to only their own models, as well as prevent them from altering multi-user settings like changing the amount of sponsored usage. They will be able to create cars, models, run evaluations and submit models to virtual races.

Join me next time where I’ll give an overview on setting up a virtual race, along with details on how our competition is going!