AWS DeepRacer League — a retrospective after the race day

AWS DeepRacer League

DeepRacer console
Virtual circuit
Community races
Get started

Make your car

Garage
DeepRacer — build a car
Action space
Action list
Customize the car

Make you model

Models
Create model — model name and environment
Create model — training type, training algorithm, and agent
Create model — customize reward function

Reward functions

def reward_function(params):
'''
Example of rewarding the agent to follow center line
'''

# Read input parameters
track_width = params['track_width']
distance_from_center = params['distance_from_center']

# Calculate 3 markers that are at varying distances away from the center line
marker_1 = 0.1 * track_width
marker_2 = 0.25 * track_width
marker_3 = 0.5 * track_width

# Give higher reward if the car is closer to center line and vice versa
if distance_from_center <= marker_1:
reward = 1.0
elif distance_from_center <= marker_2:
reward = 0.5
elif distance_from_center <= marker_3:
reward = 0.1
else:
reward = 1e-3 # likely crashed/ close to off track

return float(reward)
def reward_function(params):
'''
Example of rewarding the agent to stay inside the two borders of the track
'''

# Read input parameters
all_wheels_on_track = params['all_wheels_on_track']
distance_from_center = params['distance_from_center']
track_width = params['track_width']

# Give a very low reward by default
reward = 1e-3
# Give a high reward if no wheels go off the track and
# the agent is somewhere in between the track borders
if all_wheels_on_track and (0.5*track_width - distance_from_center) >= 0.05:
reward = 1.0
# Always return a float value
return float(reward)
def reward_function(params):
'''
Example of penalize steering, which helps mitigate zig-zag behaviors
'''

# Read input parameters
distance_from_center = params['distance_from_center']
track_width = params['track_width']
steering = abs(params['steering_angle']) # Only need the absolute steering angle
# Calculate 3 markers that are at varying distances away from the center line
marker_1 = 0.1 * track_width
marker_2 = 0.25 * track_width
marker_3 = 0.5 * track_width
# Give higher reward if the agent is closer to center line and vice versa
if distance_from_center <= marker_1:
reward = 1
elif distance_from_center <= marker_2:
reward = 0.5
elif distance_from_center <= marker_3:
reward = 0.1
else:
reward = 1e-3 # likely crashed/ close to off track
# Steering penality threshold, change the number based on your action space setting
ABS_STEERING_THRESHOLD = 15
# Penalize reward if the agent is steering too much
if steering > ABS_STEERING_THRESHOLD:
reward *= 0.8
return float(reward)
def reward_function(params):
'''
Example of rewarding the agent to stay inside two borders
and penalizing getting too close to the objects in front
'''
all_wheels_on_track = params['all_wheels_on_track']
distance_from_center = params['distance_from_center']
track_width = params['track_width']
objects_distance = params['objects_distance']
_, next_object_index = params['closest_objects']
objects_left_of_center = params['objects_left_of_center']
is_left_of_center = params['is_left_of_center']
# Initialize reward with a small number but not zero
# because zero means off-track or crashed
reward = 1e-3
# Reward if the agent stays inside the two borders of the track
if all_wheels_on_track and (0.5 * track_width - distance_from_center) >= 0.05:
reward_lane = 1.0
else:
reward_lane = 1e-3
# Penalize if the agent is too close to the next object
reward_avoid = 1.0
# Distance to the next object
distance_closest_object = objects_distance[next_object_index]
# Decide if the agent and the next object is on the same lane
is_same_lane = objects_left_of_center[next_object_index] == is_left_of_center
if is_same_lane:
if 0.5 <= distance_closest_object < 0.8:
reward_avoid *= 0.5
elif 0.3 <= distance_closest_object < 0.5:
reward_avoid *= 0.2
elif distance_closest_object < 0.3:
reward_avoid = 1e-3 # Likely crashed
# Calculate reward by putting different weights on
# the two aspects above
reward += 1.0 * reward_lane + 4.0 * reward_avoid
return reward
import mathdef reward_function(params):    
progress = params['progress']

# Read input variables
waypoints = params['waypoints']
closest_waypoints = params['closest_waypoints']
heading = params['heading']

reward = 1.0

if progress == 100:
reward += 100

# Calculate the direction of the center line based on the closest waypoints
next_point = waypoints[closest_waypoints[1]]
prev_point = waypoints[closest_waypoints[0]]
# Calculate the direction in radius, arctan2(dy, dx), the result is (-pi, pi) in radians
track_direction = math.atan2(next_point[1] - prev_point[1], next_point[0] - prev_point[0])
# Convert to degree
track_direction = math.degrees(track_direction)
# Calculate the difference between the track direction and the heading direction of the car
direction_diff = abs(track_direction - heading)
# Penalize the reward if the difference is too large
DIRECTION_THRESHOLD = 10.0

malus=1

if direction_diff > DIRECTION_THRESHOLD:
malus=1-(direction_diff/50)
if malus<0 or malus>1:
malus = 0
reward *= malus

return reward
import mathdef reward_function(params):
'''
Use square root for center line - ApiDragons-M11
'''

track_width = params['track_width']
distance_from_center = params['distance_from_center']
speed = params['speed']
progress = params['progress']
all_wheels_on_track = params['all_wheels_on_track']
SPEED_TRESHOLD = 3
reward = 1 - (distance_from_center / (track_width/2))**(4)if reward < 0:
reward = 0
if speed > SPEED_TRESHOLD:
reward *= 0.8
if not (all_wheels_on_track):
reward = 0
if progress == 100:
reward += 100
return float(reward)

Model training and evaluation

Model training and evaluation — M20
Model training and evaluation — M11
March qualifier
Rank in the community race — before racing day
Rank in the community race — racing day

--

--

Agnasarp is a technology-focused blog that has enough information about cutting-edge technologies that you can use for your problems. Stay with us!

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store