burlap.ros

Class AbstractRosEnvironment

java.lang.Object

burlap.ros.AbstractRosEnvironment

All Implemented Interfaces:

burlap.mdp.singleagent.environment.Environment

Direct Known Subclasses:

RosEnvironment

public abstract class AbstractRosEnvironment
extends Object
implements burlap.mdp.singleagent.environment.Environment

An abstract Environment implementation that executes actions on robots controlled by ROS via ROS Bridge. Note that this abstract class does not implement the Environment methods Environment.currentObservation() or Environment.isInTerminalState() and the Environment.lastReward() method requires this class's abstract method getMostRecentRewardSignal(State, Action, State) to be implemented. Instead, the primary purpose of this class is to provide a general framework for handling action execution on ROS via the ActionPublisher methodology. Specifically, for each BURLAP action (identified by its String action name), an ActionPublisher may be specified using one of the following methods:

setActionPublisher(ActionType, burlap.ros.actionpub.ActionPublisher),

setActionPublisher(String, burlap.ros.actionpub.ActionPublisher)

setActionPublisherForMultipleActions(java.util.List, burlap.ros.actionpub.ActionPublisher), or

setActionPublisherForMultipleAcitonNames(java.util.List, burlap.ros.actionpub.ActionPublisher).

When executeAction(Action) is called, the corresponding ActionPublisher for the provided Action is retrieved (with a runtime exception being thrown if none have been set for it). The ActionPublisher.publishAction(Action) is then called and passed the input Action. After the method returns, the thread sleeps (and blocks) for the amount of time in milliseconds specified by the return value of the method so long as it is greater than zero. The returned EnvironmentOutcome object is formed by querying the Environment.currentObservation() method before the ActionPublisher is invoked for the pre-state and after the thread sleeping completes for the post-state; this object's abstract getMostRecentRewardSignal(State, Action, State) method is then called to get the reward signal to use (and is provided the state-action-state transition that was just recorded, which may or may not be needed); and the the terminal state flag is set by invoking this object's Environment.isInTerminalState(). If the environment transitions to a terminal state, then before the executeAction(Action) method exits, it calls the handleEnterTerminalState() abstract method. This latter method does not have to be implemented, but allows you to easily inject code for handling the special case when the robot enters a terminal state. Therefore, as long as all necessary methods for this abstract class are implemented, everything will work.

Author:

James MacGlashan.

Field Summary

Fields

Modifier and Type	Field and Description
protected Map<String,ActionPublisher>	actionPublishers The mapping from BURLAP Action names to the corresponding ActionPublisher to use.
protected double	lastReward The last reward signal generated.
protected ros.RosBridge	rosBridge The RosBridge connection

Constructor Summary

Constructors

Constructor and Description
AbstractRosEnvironment(ros.RosBridge rosBridge) Initializes with the given RosBridge.
AbstractRosEnvironment(String rosBridgeURI) Creates a RosBridge connection for this Environment at the given ROS Bridge URI.

Method Summary

Modifier and Type	Method and Description
burlap.mdp.singleagent.environment.EnvironmentOutcome	executeAction(burlap.mdp.core.action.Action a)
protected abstract double	getMostRecentRewardSignal(burlap.mdp.core.state.State s, burlap.mdp.core.action.Action a, burlap.mdp.core.state.State sprime) Generates the reward signal for the most recent environment interaction initiated through the executeAction(Action) method and what will be returned by the lastReward() method until another executeAction(Action) method is invoked or this environment is reset with resetEnvironment().
ros.RosBridge	getRosBridge() Returns the RosBridge object to which this environment is connected.
protected abstract void	handleEnterTerminalState() This method is called just before exiting the executeAction(Action) method if the newly entered environment state is a terminal state.
double	lastReward()
void	resetEnvironment()
void	setActionPublisher(burlap.mdp.core.action.ActionType action, ActionPublisher ap) Sets the ActionPublisher to handle executions of the given ActionType assuming that Actions generated by the ActionType have the same name as the type name.
void	setActionPublisher(String actionName, ActionPublisher ap) Sets the ActionPublisher to handle executions of actions with the name actionName
void	setActionPublisherForMultipleAcitonNames(List<String> actionNames, ActionPublisher ap) Sets a single ActionPublisher to handle the execution of a list of actions identified by given action names.
void	setActionPublisherForMultipleActions(List<burlap.mdp.core.action.ActionType> actions, ActionPublisher ap) Sets a single ActionPublisher to handle the execution of a list of ActionType objects.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface burlap.mdp.singleagent.environment.Environment

currentObservation, isInTerminalState

Field Detail

rosBridge

protected ros.RosBridge rosBridge

The RosBridge connection

actionPublishers

protected Map<String,ActionPublisher> actionPublishers

The mapping from BURLAP Action names to the corresponding ActionPublisher to use.

lastReward

protected double lastReward

The last reward signal generated.

Constructor Detail

AbstractRosEnvironment

public AbstractRosEnvironment(ros.RosBridge rosBridge)

Initializes with the given RosBridge.

Parameters:

rosBridge - the RosBridge connection to use.

AbstractRosEnvironment

public AbstractRosEnvironment(String rosBridgeURI)

Creates a RosBridge connection for this Environment at the given ROS Bridge URI. and blocks until a connection with has been established.

Parameters:

rosBridgeURI - the ROS Bridge URI; e.g., ws://localhost:9090

Method Detail

setActionPublisher

public void setActionPublisher(String actionName,
                               ActionPublisher ap)

Sets the ActionPublisher to handle executions of actions with the name actionName

Parameters:

actionName - the name of the action

ap - the ActionPublisher that handles executions of actions with that name

setActionPublisher

public void setActionPublisher(burlap.mdp.core.action.ActionType action,
                               ActionPublisher ap)

Sets the ActionPublisher to handle executions of the given ActionType assuming that Actions generated by the ActionType have the same name as the type name.

Parameters:

action - the ActionType to handle

ap - the ActionPublisher to handle the executions

setActionPublisherForMultipleActions

public void setActionPublisherForMultipleActions(List<burlap.mdp.core.action.ActionType> actions,
                                                 ActionPublisher ap)

Sets a single ActionPublisher to handle the execution of a list of ActionType objects. Assumes that Actions generated by the ActionTypes will have the same name as the name of the types that generated them.

Parameters:

actions - the ActionType objects to handle

ap - the ActionPublisher that handles the execution

setActionPublisherForMultipleAcitonNames

public void setActionPublisherForMultipleAcitonNames(List<String> actionNames,
                                                     ActionPublisher ap)

Sets a single ActionPublisher to handle the execution of a list of actions identified by given action names.

Parameters:

actionNames - the list of action names to handle

ap - the ActionPublisher that handles execution

getRosBridge

public ros.RosBridge getRosBridge()

Returns the RosBridge object to which this environment is connected.

Returns:

the RosBridge object to which this environment is connected.

executeAction

public burlap.mdp.singleagent.environment.EnvironmentOutcome executeAction(burlap.mdp.core.action.Action a)

Specified by:

executeAction in interface burlap.mdp.singleagent.environment.Environment

lastReward

public double lastReward()

Specified by:

lastReward in interface burlap.mdp.singleagent.environment.Environment

resetEnvironment

public void resetEnvironment()

Specified by:

resetEnvironment in interface burlap.mdp.singleagent.environment.Environment

getMostRecentRewardSignal

protected abstract double getMostRecentRewardSignal(burlap.mdp.core.state.State s,
                                                    burlap.mdp.core.action.Action a,
                                                    burlap.mdp.core.state.State sprime)

Generates the reward signal for the most recent environment interaction initiated through the executeAction(Action) method and what will be returned by the lastReward() method until another executeAction(Action) method is invoked or this environment is reset with resetEnvironment().

Parameters:

s - the previous environment state before this reward event

a - the action taken by the agent before this reward event

sprime - the resulting environment state associated with this reward event

Returns:

the reward for the most recent environment interaction

handleEnterTerminalState

protected abstract void handleEnterTerminalState()

This method is called just before exiting the executeAction(Action) method if the newly entered environment state is a terminal state. This method does not have to do anything, but is useful if you need to do something special with the robot when it reaches a terminal state.