burlap.ros

Class RosEnvironment

java.lang.Object

burlap.ros.AbstractRosEnvironment

burlap.ros.RosEnvironment

All Implemented Interfaces:

burlap.mdp.singleagent.environment.Environment, ros.RosListenDelegate

public abstract class RosEnvironment
extends AbstractRosEnvironment
implements ros.RosListenDelegate

An abstract implementation of AbstractRosEnvironment that maintains the environment state by receiving a single ROS message corresponding to the state of the environment. To implement this class, implement the unpackStateFromMsg(JsonNode, String) method to provide the code that parses the ROS message into a BURLAP State.

Note that in the constructor you may want to set a low (e.g., 1) throttle rate and queue rate if state messages are sent frequently, otherwise ROS Bridge may start lagging.

This class will block on the currentObservation() method until it receives a state message from ROS Bridge. If you would like, you can override waiting for the first state message received from ROS and force this environment to report a specific BURLAP State using the overrideFirstReceivedState(State) method. Note that any subsequent burlap_msgs/burlap_state messages will still update this environment's current state.

This class may also be provided a BURLAP RewardFunction and TerminalFunction for generating non-zero rewards and terminal states. Use the setRewardFunction(RewardFunction) and setTerminalFunction(TerminalFunction) to set them use the setRewardFunction(RewardFunction) and setTerminalFunction(TerminalFunction) methods. This class does not have the handleEnterTerminalState() inherited from AbstractRosEnvironment do anything, so if you want to inject code for handling the event when the environment enters a terminal state, you should subclass this class and override that method.

Author:

James MacGlashan.

Nested Class Summary

Nested classes/interfaces inherited from interface ros.RosListenDelegate

ros.RosListenDelegate.LegacyFormat

Field Summary

Fields

Modifier and Type	Field and Description
protected burlap.mdp.core.state.State	curState The current State representation of the environment
protected int	debugCode The debug code used for debug prints.
protected burlap.mdp.singleagent.SADomain	domain The BURLAP SADomain into which states will be parsed
protected boolean	printStateAsReceived Debug flag indicating whether states should be printed to the terminal as they are received.
protected Boolean	receivedFirstState Indicates whether the first state message from ROS has been received yet
protected burlap.mdp.singleagent.model.RewardFunction	rf The optional RewardFunction used to generate reward signals
protected burlap.mdp.core.TerminalFunction	tf The optional TerminalFunction used to specify terminal states of the environment

Fields inherited from class burlap.ros.AbstractRosEnvironment

actionPublishers, lastReward, rosBridge

Constructor Summary

Constructors

Constructor and Description
RosEnvironment(burlap.mdp.singleagent.SADomain domain, ros.RosBridge ros, String rosStateTopic, int rosBridgeThrottleRate, int rosBridgeQueueLength) Creates an environment wrapper for state information provided over ROS with BURLAP actions needing to be published to ROS.
RosEnvironment(burlap.mdp.singleagent.SADomain domain, ros.RosBridge ros, String rosStateTopic, String rosStateMessageType) Creates an environment wrapper for state information provided over ROS with BURLAP actions needing to be published to ROS.
RosEnvironment(burlap.mdp.singleagent.SADomain domain, ros.RosBridge ros, String rosStateTopic, String rosStateMessageType, int rosBridgeThrottleRate, int rosBridgeQueueLength) Creates an environment wrapper for state information provided over ROS with BURLAP actions needing to be published to ROS.
RosEnvironment(burlap.mdp.singleagent.SADomain domain, String rosBridgeURI, String rosStateTopic, String rosStateMessageType) Creates an environment wrapper for state information provided over ROS with BURLAP actions needing to be published to ROS.
RosEnvironment(burlap.mdp.singleagent.SADomain domain, String rosBridgeURI, String rosStateTopic, String rosStateMessageType, int rosBridgeThrottleRate, int rosBridgeQueueLength) Creates an environment wrapper for state information provided over ROS with BURLAP actions needing to be published to ROS.

Method Summary

Modifier and Type	Method and Description
void	blockUntilStateReceived() A method you can call that forces the calling thread to wait until the first state from ROS has been received.
burlap.mdp.core.state.State	currentObservation()
burlap.mdp.singleagent.SADomain	getDomain() Returns the domain of this environment.
protected double	getMostRecentRewardSignal(burlap.mdp.core.state.State s, burlap.mdp.core.action.Action a, burlap.mdp.core.state.State sprime) Generates the reward signal for the most recent environment interaction initiated through the AbstractRosEnvironment.executeAction(Action) method and what will be returned by the AbstractRosEnvironment.lastReward() method until another AbstractRosEnvironment.executeAction(Action) method is invoked or this environment is reset with AbstractRosEnvironment.resetEnvironment().
protected void	handleEnterTerminalState() This method is called just before exiting the AbstractRosEnvironment.executeAction(Action) method if the newly entered environment state is a terminal state.
boolean	isInTerminalState()
protected burlap.mdp.core.state.State	onStateReceive(burlap.mdp.core.state.State s) This method is called after a State is unpacked from a ROS message.
void	overrideFirstReceivedState(burlap.mdp.core.state.State s) Use this method to override the first received state from ROS to be the provided State.
void	receive(com.fasterxml.jackson.databind.JsonNode data, String stringRep)
void	setPrintStateAsReceived(boolean printStateAsReceived) Sets whether the default implementation of onStateReceive(State) will print the state information to the terminal.
void	setRewardFunction(burlap.mdp.singleagent.model.RewardFunction rf) Sets a BURLAP RewardFunction to use to provide reward signals from the AbstractRosEnvironment.lastReward() method.
void	setTerminalFunction(burlap.mdp.core.TerminalFunction tf) Sets the BURLAP TerminalFunction to use to provide terminal state checks from the isInTerminalState().
abstract burlap.mdp.core.state.State	unpackStateFromMsg(com.fasterxml.jackson.databind.JsonNode data, String stringRep) This method receive the RosBridge message and unpacks it into a BURLAP State object.

Methods inherited from class burlap.ros.AbstractRosEnvironment

executeAction, getRosBridge, lastReward, resetEnvironment, setActionPublisher, setActionPublisher, setActionPublisherForMultipleAcitonNames, setActionPublisherForMultipleActions

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

domain

protected burlap.mdp.singleagent.SADomain domain

The BURLAP SADomain into which states will be parsed

curState

protected burlap.mdp.core.state.State curState

The current State representation of the environment

rf

protected burlap.mdp.singleagent.model.RewardFunction rf

The optional RewardFunction used to generate reward signals

tf

protected burlap.mdp.core.TerminalFunction tf

The optional TerminalFunction used to specify terminal states of the environment

receivedFirstState

protected Boolean receivedFirstState

Indicates whether the first state message from ROS has been received yet

printStateAsReceived

protected boolean printStateAsReceived

Debug flag indicating whether states should be printed to the terminal as they are received. Default value is false.

debugCode

protected int debugCode

The debug code used for debug prints.

Constructor Detail

RosEnvironment

public RosEnvironment(burlap.mdp.singleagent.SADomain domain,
                      String rosBridgeURI,
                      String rosStateTopic,
                      String rosStateMessageType)

Creates an environment wrapper for state information provided over ROS with BURLAP actions needing to be published to ROS.

Remember that for actions to be properly handled, you will need to set the ActionPublisher to use for each action after this class is constructed with one of the appropriate methods (e.g., AbstractRosEnvironment.setActionPublisher(String, burlap.ros.actionpub.ActionPublisher)).

Parameters:

domain - the domain into which ROS burlap_state messages are parsed

rosBridgeURI - the URI of the ros bridge server. Note that by default, ros bridge uses port 9090. An example URI is ws://localhost:9090

rosStateTopic - the name of the ROS topic that publishes the state messages.

rosStateMessageType - the message type of the ROS state messages.

RosEnvironment

public RosEnvironment(burlap.mdp.singleagent.SADomain domain,
                      String rosBridgeURI,
                      String rosStateTopic,
                      String rosStateMessageType,
                      int rosBridgeThrottleRate,
                      int rosBridgeQueueLength)

Creates an environment wrapper for state information provided over ROS with BURLAP actions needing to be published to ROS.

Remember that for actions to be properly handled, you will need to set the ActionPublisher to use for each action after this class is constructed with one of the appropriate methods (e.g., AbstractRosEnvironment.setActionPublisher(String, burlap.ros.actionpub.ActionPublisher)).

Parameters:

domain - the domain into which ROS burlap_state messages are parsed

rosBridgeURI - the URI of the ros bridge server. Note that by default, ros bridge uses port 9090. An example URI is ws://localhost:9090

rosStateTopic - the name of the ROS topic that publishes the state messages.

rosStateMessageType - the message type of the ROS state messages.

rosBridgeThrottleRate - the ROS Bridge server throttle rate: how frequently the server will send state messages

rosBridgeQueueLength - the ROS Bridge queue length: how many messages are queued on the server; queueing is a consequence of the throttle rate

RosEnvironment

public RosEnvironment(burlap.mdp.singleagent.SADomain domain,
                      ros.RosBridge ros,
                      String rosStateTopic,
                      String rosStateMessageType)

Creates an environment wrapper for state information provided over ROS with BURLAP actions needing to be published to ROS. The state message type is assumed to be burlap_msgs/burlap_state.

Remember that for actions to be properly handled, you will need to set the ActionPublisher to use for each action after this class is constructed with one of the appropriate methods (e.g., AbstractRosEnvironment.setActionPublisher(String, burlap.ros.actionpub.ActionPublisher)).

Parameters:

domain - the domain into which ROS burlap_state messages are parsed

ros - the connected RosBridge instance.

rosStateTopic - the name of the ROS topic that publishes the burlap_msgs/burlap_state messages.

rosStateMessageType - the message type of the ROS state messages.

RosEnvironment

public RosEnvironment(burlap.mdp.singleagent.SADomain domain,
                      ros.RosBridge ros,
                      String rosStateTopic,
                      int rosBridgeThrottleRate,
                      int rosBridgeQueueLength)

Creates an environment wrapper for state information provided over ROS with BURLAP actions needing to be published to ROS. The message type is assumed to be burlap_msgs/burlap_state.

Remember that for actions to be properly handled, you will need to set the ActionPublisher to use for each action after this class is constructed with one of the appropriate methods (e.g., AbstractRosEnvironment.setActionPublisher(String, burlap.ros.actionpub.ActionPublisher)).

Parameters:

domain - the domain into which ROS burlap_state messages are parsed

ros - the connected RosBridge instance.

rosStateTopic - the name of the ROS topic that publishes the burlap_msgs/burlap_state messages.

rosBridgeThrottleRate - the ROS Bridge server throttle rate: how frequently the server will send state messages

rosBridgeQueueLength - the ROS Bridge queue length: how many messages are queued on the server; queueing is a consequence of the throttle rate

RosEnvironment

public RosEnvironment(burlap.mdp.singleagent.SADomain domain,
                      ros.RosBridge ros,
                      String rosStateTopic,
                      String rosStateMessageType,
                      int rosBridgeThrottleRate,
                      int rosBridgeQueueLength)

Creates an environment wrapper for state information provided over ROS with BURLAP actions needing to be published to ROS. Here, the state message type may be specified to be something other than burlap_msgs/burlap_state. Note that unless you have subclassed RosEnvironment and overridden unpackStateFromMsg(JsonNode, String), the message type should be a type that is or adheres to "burlap_msgs/burlap_state" even if named something else.

Remember that for actions to be properly handled, you will need to set the ActionPublisher to use for each action after this class is constructed with one of the appropriate methods (e.g., AbstractRosEnvironment.setActionPublisher(String, burlap.ros.actionpub.ActionPublisher)).

Parameters:

domain - the domain into which ROS burlap_state messages are parsed

ros - the connected RosBridge instance.

rosStateTopic - the name of the ROS topic that publishes the state messages.

rosStateMessageType - the message type of the ROS state messages.

rosBridgeThrottleRate - the ROS Bridge server throttle rate: how frequently the server will send state messages

rosBridgeQueueLength - the ROS Bridge queue length: how many messages are queued on the server; queueing is a consequence of the throttle rate

Method Detail

setRewardFunction

public void setRewardFunction(burlap.mdp.singleagent.model.RewardFunction rf)

Sets a BURLAP RewardFunction to use to provide reward signals from the AbstractRosEnvironment.lastReward() method.

Parameters:

rf - RewardFunction to use to provide reward signals.

setTerminalFunction

public void setTerminalFunction(burlap.mdp.core.TerminalFunction tf)

Sets the BURLAP TerminalFunction to use to provide terminal state checks from the isInTerminalState().

Parameters:

tf - the BURLAP TerminalFunction to use to provide terminal state checks

getDomain

public burlap.mdp.singleagent.SADomain getDomain()

Returns the domain of this environment.

Returns:

the domain of this environment.

currentObservation

public burlap.mdp.core.state.State currentObservation()

Specified by:

currentObservation in interface burlap.mdp.singleagent.environment.Environment

isInTerminalState

public boolean isInTerminalState()

Specified by:

isInTerminalState in interface burlap.mdp.singleagent.environment.Environment

getMostRecentRewardSignal

protected double getMostRecentRewardSignal(burlap.mdp.core.state.State s,
                                           burlap.mdp.core.action.Action a,
                                           burlap.mdp.core.state.State sprime)

Description copied from class: AbstractRosEnvironment

Generates the reward signal for the most recent environment interaction initiated through the AbstractRosEnvironment.executeAction(Action) method and what will be returned by the AbstractRosEnvironment.lastReward() method until another AbstractRosEnvironment.executeAction(Action) method is invoked or this environment is reset with AbstractRosEnvironment.resetEnvironment().

Specified by:

getMostRecentRewardSignal in class AbstractRosEnvironment

Parameters:

s - the previous environment state before this reward event

a - the action taken by the agent before this reward event

sprime - the resulting environment state associated with this reward event

Returns:

the reward for the most recent environment interaction

setPrintStateAsReceived

public void setPrintStateAsReceived(boolean printStateAsReceived)

Sets whether the default implementation of onStateReceive(State) will print the state information to the terminal.

Parameters:

printStateAsReceived - if true, then the default implementation of onStateReceive(State) will print states to the screen; if false, the state receiving is silent by default.

overrideFirstReceivedState

public void overrideFirstReceivedState(burlap.mdp.core.state.State s)

Use this method to override the first received state from ROS to be the provided State. If the blockUntilStateReceived() method has been called in another thread, it will be notified that the first state is received. Note that if ROS Bridge subsequently sends state message, the state of this environment will be updated to reflect them.

Parameters:

s - the State to force this Environment to.

blockUntilStateReceived

public void blockUntilStateReceived()

A method you can call that forces the calling thread to wait until the first state from ROS has been received.

receive

public void receive(com.fasterxml.jackson.databind.JsonNode data,
                    String stringRep)

Specified by:

receive in interface ros.RosListenDelegate

unpackStateFromMsg

public abstract burlap.mdp.core.state.State unpackStateFromMsg(com.fasterxml.jackson.databind.JsonNode data,
                                                               String stringRep)

This method receive the RosBridge message and unpacks it into a BURLAP State object.

Parameters:

data - the JSON message.

stringRep - the string representation of the JSON message.

Returns:

a State object that is unpacked from the JSON message.

onStateReceive

protected burlap.mdp.core.state.State onStateReceive(burlap.mdp.core.state.State s)

This method is called after a State is unpacked from a ROS message. The new current state of the environment will be set to whatever this method returns. By default, this method simply returns the same reference and if this environment's printStateAsReceived data member is set to true, then it will print to the terminal the string representation of the state. Override this method to provide special handling of used state (e.g., adding virtual objects to the state that ROS does not perceive).

Parameters:

s - the parsed state from the ROS message received.

Returns:

returns the State object parsed from the ros message

handleEnterTerminalState

protected void handleEnterTerminalState()

Description copied from class: AbstractRosEnvironment

This method is called just before exiting the AbstractRosEnvironment.executeAction(Action) method if the newly entered environment state is a terminal state. This method does not have to do anything, but is useful if you need to do something special with the robot when it reaches a terminal state.

Specified by:

handleEnterTerminalState in class AbstractRosEnvironment