First download the simulator and unzip it.
You just need to run the appropriate executable file in windows or Linux.
In windows: Run the GrabSimClient.exe executable.
In Linux:
bash /path/to/LinuxClient/GrabSimClient.sh
Devices without graphical interfaces [1]
Verify that you can access the Unreal Engine source code repository on GitHub: https://github.com/EpicGames/UnrealEngine. If you cannot access the repository then you will need to link your GitHub account with your Epic Games Account.
To download container images from GitHub Container Registry using Docker
you will need to authenticate using a personal access token. If you do
not already have a personal access token with the read:packages
scope then you will need to follow the steps to create
one.
Once you have created a personal access token with the required scope,
use the docker login
command to authenticate with GitHub Container
Registry as described in the instructions from
GitHub.
This command will need to be run from the command-line interface。
Once you have opened the command-line prompt then run the command shown
below, replacing ACCESS_TOKEN
with your personal access token and
USERNAME
with your GitHub username:
echo ACCESS_TOKEN | docker login ghcr.io -u USERNAME --password-stdin
If the authentication process was successful then you should see the message “Login Succeeded” displayed.
The official prebuilt container images for Unreal Engine are stored as
image tags in the
ghcr.io/epicgames/unreal-engine
repository. To download the Linux development image for Unreal Engine
4.27, use the docker pull
command shown below:
docker pull ghcr.io/epicgames/unreal-engine:dev-4.27
This will download a container image that encapsulates the files for Unreal Editor and build tools, which are quite large. Depending on the speed of your internet connection, the download process may take some time. When the download is complete, you should see the message “Status: Downloaded newer image for ghcr.io/epicgames/unreal-engine:dev-4.27” displayed.
You need to make sure you have sufficient permissions on the LinuxClient:
cd LinuxClient chmod -R 775 Engine chmod -R 775 GrabSim chmod 777 GrabSimClient.sh
Use the following docker file code to create a new one to run.
FROM ghcr.io/epicgames/unreal-engine:runtime-pixel-streaming COPY --chown=ue5:ue5 LinuxClient path/to/LinuxClient ENV NVIDIA_DRIVER_CAPABILITIES all ENTRYPOINT ["bin/bash", "/path/to/LinuxClient/GrabSimClient.sh", "-RenderOffScreen"] CMD []
After creating the container you can enter as root if needed:
docker exec -it -u root YOUR_CONTAINER /bin/bash
To exit the container, run the command shown below:
exit
Note that our simulator runs in a docker container and occupies port 30001. If you want to map to other ports(eg. port 30020) on the host and control the simulator on the server, you need to build the container with the following command:
nvidia-docker run --privileged -it --gpus='"device=1"' -p 30020:30001 --name sim30020 -v /YOUR/Mount/FOLDER:/home YOUR_BUILDED_IMAGE /bin/bash cd /PATH/TO/LinuxClient bash GrabSim.sh -RenderOffScreen
- Protocol: gRPC
- Port: 30001
Only the data type is introduced, please refer to the interface method section for interface details.
Used for initword( In version 1)
F i e l d | T y p e | V a l u e | Description |
---|---|---|---|
v a l u e | i n t 3 2 | ( 0 ) | Number of scenes in world. Means num_processes. Usually set to 1 |
Usage:
GrabSim_pb2.Count(value=1)
Used for initword( In version 2)
F i e l d | T y p e | Va lue | Description |
---|---|---|---|
c o u n t | i n t 3 2 | Number of scenes in world. Means num_processes. Usually set to 1 | |
m a p I D | i n t 3 2 | 3 ,4, 5 | Meaning of mapID: 3 : Coffee 4 : Restaurant 5 : Nurse home |
Usage:
GrabSim_pb2.BatchMap(count=1, mapID=3)
No content, used when interface does not need input or output values.( in version 1)
Usage:
GrabSim_pb2.Nothing()
No content, used when interface does not need input or output values. Equal to the Data Type: Nothing. (in version 2)
Usage:
GrabSim_pb2.NUL()
F i e l d | T y p e | V a l u e | Description |
---|---|---|---|
v a l u e | i n t 3 2 | ( 0 ) | Scene ID. The desirable range is [0, Count-1]. Default is 0 |
Usage:
GrabSim_pb2.SceneID(value=0)
Field | Type | Description |
---|---|---|
scenes | list/Scene | All scenes in world |
error | string | Partial error information from execution |
Field | Type | Value | Description |
---|---|---|---|
scene | int32 | Target scene ID | |
adjust | bool | (False) | Set to True for init params to take effect |
height | float | 78.5~111.5 | (90.4) Table height |
width | float | 50~150 | (107.4) Table width |
Usage:
GrabSim_pb2.ResetParams()
Field | Type | Description |
---|---|---|
name | string | Object name |
location | Location | Object position |
rotation | Rotation | Object rotation |
Usage:
GrabSim_pb2.Object(name = "AA",type = "ADMilk",location = GrabSim_pb2.Location(X=30,Y=-260,Z=84))
Field | Type | Description |
---|---|---|
X | float | X coordinate |
Y | float | Y coordinate |
Z | float | Z coordinate |
Usage:
GrabSim_pb2.Location(X=30,Y=-260,Z=84)
Field | Type | Description |
---|---|---|
angle | float | Rotation angle in degrees |
F ield | Type | V alue | Description |
---|---|---|---|
s cene | int32 | Target scene ID | |
ap pend | bool | (Fa lse) | Set to append objects or clear existing ones |
obj ects | lis t/Object | List of objects |
Field | Type | Value | Description |
---|---|---|---|
x, y | float | Object position, height at table top | |
type | int | Object ID |
Field | Type | Value | Description |
---|---|---|---|
scene | int32 | Target scene ID | |
objectIDs | list/int32 | Index of objects in Scene.Objects |
F i e l d | Ty pe | Value | Description |
---|---|---|---|
s c e n e | i nt 32 | Target scene ID | |
a c t i o n | en um | Gr abSim_pb2.Action.ActionType. WalkToGrabSim_pb2.Action.Act ionType.GraspGrabSim_pb2.Act ion.ActionType.ReleaseGrabSi m_pb2.Action.ActionType.Move | WalkTo: Adjust robot position, 5 params Grasp: Control grasping, left/right hand in valuesRelease: Control releasing, left/right hand in values Move: Control joint angles, 21 params in values |
v a l u e s | li st /f lo at | ([0, …]) | WalkTo: Adjust robot position, 5 paramsX, Y, Yaw, 0/-1/1, distance: If it cannot be reached, it will find the target within 10 cm# 0: Query only, not move # -1: Teleport to target position # 1: Navigation to target position |
Usage:
GrabSim_pb2.Action( scene=0, action=GrabSim_pb2.Action.ActionType.WalkTo, values=[location[0], location[1], location[2], -1, 10] )
Field | Type | Description |
---|---|---|
name | string | Joint name |
location | Location | Joint position |
rotation | Rotation | Joint rotation |
Usage:
GrabSim_pb2.Joint(name="joint1", location=Location(1.0, 2.0, 3.0), rotation=Rotation(45))
F ield | Type | Description |
---|---|---|
sce neID | int32 | Scene ID |
loca tion | Location | Robot coordinates (center of workspace, Scene coordinate system) |
rota tion | Rotation | Robot rotation angles |
jo ints | list/ Scene.Joint | Pose information for robot joints |
fin gers | list/S cene.Finger | Pose information for robot finger joints |
obj ects | list/S cene.Object | Position and info of all objects in scene. First object is table, last few are hands with no position info |
t imes tamp | int64 | Nanoseconds since 1970/1/1 |
e rror | string | Partial error information from execution |
Usage:
import numpy as np p_x, p_y = scene.location.X, scene.location.Y yaw = scene.rotation.Yaw * np.pi / 180
scene = stub.Observe(GrabSim_pb2.SceneID(value=0)) print('------------------show_env_info----------------------') print( f"location:{[scene.location.X, scene.location.Y]}, rotation:{scene.rotation.Yaw}\n", f"joints number:{len(scene.joints)}, fingers number:{len(scene.fingers)}\n", f"objects number: {len(scene.objects)}\n" f"rotation:{scene.rotation}, timestep:{scene.timestep}\n" f"timestamp:{scene.timestamp}, collision:{scene.collision}, info:{scene.info}")
Field | Type | Description |
---|---|---|
name | string | Joint name |
location | Location | Joint position |
angle | float | Joint angle |
Field | Type | Description |
---|---|---|
name | string | Finger name |
location | list/Location | Position of each joint of the finger |
angle | float | Joint angle |
Field | Type | Description |
---|---|---|
name | string | Object name |
location | Location | Object position |
rotation | Rotation | Object rotation angle (-180, 180) |
boxes | list/Object.Box.Diagonal | Bounding boxes of object |
Usage:
GrabSim_pb2.Scene.Object([{'name': 'NFCJuice', 'location': [2525, 2510], 'angle': [0, 0, 0]}])
Field | Type | Description |
---|---|---|
timestamp | int64 | Timestamp in nanoseconds |
joints | listJoint | Robot joint poses |
Usage:
GrabSim_pb2.Pose(X=loc[0], Y=loc[1], Yaw=180)
F ield | Type | V alue | Description |
---|---|---|---|
s cene | int32 | Target scene ID | |
ap pend | bool | (Fa lse) | Set to append objects or clear existing ones |
obj ects | lis t/Object | List of objects |
Field | Type | Value | Description |
---|---|---|---|
x, y | float | Object position, height at table top | |
type | int | Object ID |
Field | Type | Value | Description |
---|---|---|---|
sceneID | int32 | Target scene ID | |
objectIDs | list/int32 | Index of objects in Scene.Objects |
Field | Type | Description |
---|---|---|
x | float | Robot x coordinate |
y | float | Robot y coordinate |
angle | float | Robot current angle |
speed | float | Robot moving speed in facing direction, cm/s |
rotating | float | Robot rotation speed, degrees/s, positive is clockwise |
Field | Type | Description |
---|---|---|
sceneID | int32 | Target scene ID |
cameras | list/enum | CameraName |
CameraName:
Head_Color: Head RGB camera
Head_Depth: Head depth camera
Head_Segment: Head Segment camera
Chest_Color: Chest RGB camera
Waist_Color: Waist RGB camera
Waist_Depth: Waist depth camera
Usage:
GrabSim_pb2.CameraList(scene=0, cameras=[ GrabSim_pb2.CameraName.Head_Depth, GrabSim_pb2.CameraName.Head_Color, GrabSim_pb2.CameraName.Head_Segment ])
Field | Type | Description |
---|---|---|
images | list/CameraData.Image | Image data |
timestamp | int64 | Nanoseconds since 1970/1/1 |
Field | Type | Description |
---|---|---|
name | string | Camera name |
data | bytes | Byte array |
dtype | string | Data format (uint8, float16, etc) |
location | Location | Camera position |
rotation | Rotation | Camera rotation angles |
width | int | Image width |
height | int | Image height |
channels | int | Number of channels |
parameters | CamaraData.Image.Parameters | Camera intrinsics |
F ie ld | Type | Description |
---|---|---|
fx | float | |
fy | float | |
cx | float | |
cy | float | |
ma tr ix | arra y/float | Transform matrix from camera to robot coordinates (4x4, flattened) |
Field | Type | Description |
---|---|---|
walkers | list/enum | walker_list can be appended through WalkerList.Walker |
scene | int32 | Target scene ID |
Usage:
walker_list.append(GrabSim_pb2.WalkerList.Walker(id=i, pose=GrabSim_pb2.Pose(X=loc[0], Y=loc[1], Yaw=90))) GrabSim_pb2.WalkerList(walkers=walker_list, scene=0)
Field | Type | Description |
---|---|---|
id | int32 | The serial number of the walker to join. From 0 |
pose | GrabSim_pb2.Pose | X, Y, Yaw |
Usage:
GrabSim_pb2.WalkerList.Walker(id=i, pose=GrabSim_pb2.Pose(X=loc[0], Y=loc[1], Yaw=90))
F ield | Type | Description |
---|---|---|
cont rols | list /enum | controls_list can be appended through WalkerControls.WControl |
s cene | int32 | Target scene ID |
Usage:
controls.append(GrabSim_pb2.WalkerControls.WControl(id=i, autowalk=is_autowalk, speed=200, pose=pose)) GrabSim_pb2.WalkerControls(controls=controls, scene=0)
Field | Type | Description |
---|---|---|
id | int32 | The joined walker’s serial number. |
autowalk | str | Usually set to is_autowalk. |
speed | int32 | The speed to move. (cm/s) |
pose | GrabSim_pb2.Pose | X, Y, Yaw |
Usage:
GrabSim_pb2.WalkerControls.WControl(id=i, autowalk=is_autowalk, speed=200, pose=pose)
Field | Type | Description |
---|---|---|
IDs | list/enum | The joined walker’s serial number. |
scene | int32 | Target scene ID |
Usage:
GrabSim_pb2.RemoveList(IDs=[1, 3], scene=scene_id)
Install grpc environment
pip install grpcio
You need to download GrabSim_pb2_grpc.py and GrabSim_pb2.py corresponding to the simulator version. Then import them.
import grpc
import GrabSim_pb2_grpc
import GrabSim_pb2
Define information such as ports to communicate with the simulator. If you run the simulator on other machines, please change localhost to the IP address of the machine, and make sure your machine can access port 30001.
channel = grpc.insecure_channel('localhost:30001',options=[
('grpc.max_send_message_length', 1024*1024*1024),
('grpc.max_receive_message_length', 1024*1024*1024)
])
If you mapped the port before (e.g. mapped to port 30020), you can run directly on the server without entering the container (but keep the container running):
channel = grpc.insecure_channel('localhost:30020',options=[
('grpc.max_send_message_length', 1024*1024*1024),
('grpc.max_receive_message_length', 1024*1024*1024)
])
Create a client。
Request: Channel
Response: Scene - initial scene state
Usage:
stub = GrabSim_pb2_grpc.GrabSimStub(channel)
Initialize the simulation environment. ( In version 1)
Request: Channel
Response: Scene - initial scene state
Usage:
initworld = stub.Init(GrabSim_pb2.Count(value=1))
scene = initworld.scenes[0]
Initialize the simulation environment. (In version 2)
Request: count - number of scenes to initialize.
Meaning of mapID:
3 : Coffee
4 : Restaurant
5 : Nurse home
Response: Scene - initial scene state
Usage:
initworld = stub.SetWorld(GrabSim_pb2.BatchMap(count=1, mapID=4))
scene = initworld.scenes[0]
your can get the object type in the scenes
obj_list = set() area_list = set() for i in range(len(scene.objects)): object = scene.objects[i] name = object.name if 'Room' in name: obj_list.add(name) else: area_list.add(name) print(obj_list) print(area_list)
Reset a scene.
Request: ResetParams
Response: Scene - reset scene state
Usage:
reset_response = stub.Reset(GrabSim_pb2.ResetParams(scene=0))
Get object and robot poses in the scene.
Request: SceneID
Response: Scene
Usage:
scene = stub.Observe(GrabSim_pb2.SceneID(value=0))
objects = scene.objects
Get the position and angle of each joint of the robot
Request: SceneID
Response: Scene
Usage:
pose = stub.ObservePose(GrabSim_pb2.SceneID(value=0))
There are 3 cameras on the head of the robot in the simulator, which are depth and segmentation cameras. You can also specify the cameras of other parts of the robot (eg. Chest_Color/Waist_Color/Waist_Depth). See the Data Types section/CamerList for details.
Request: SceneID
Response: Scene
Usage:
images = stub.Capture(GrabSim_pb2.CameraList(sceneID=0, cameras=[ GrabSim_pb2.CameraName.Head_Depth, GrabSim_pb2.CameraName.Head_Color, GrabSim_pb2.CameraName.Head_Segment ])).images depth = np.frombuffer(images[0].data, dtype=images[0].dtype).reshape( (images[0].height, images[0].width, images[0].channels)) rgb = np.frombuffer(images[1].data, dtype=images[1].dtype).reshape( (images[1].height, images[1].width, images[1].channels)) # convert to BGR format rgb = rgb[:, :, [2, 1, 0]]
Execute an action in the scene. Support WalkTo, Grasp, Release, Move actions. See the Data Types section/Action for details.
Flexible angle and can walk to any reachable area. In VLN tasks, when executing the predicted action, you can customize the rotation angle and displacement distance corresponding to each action.
Request: Action
Response: Scene - updated scene state
Usage:
Navigate to (-650.0, -1550.0, -15.0), if you can’t reach it, you will find the target within 10 cm
Scene = stub.Do(GrabSim_pb2.Action(
scene=0,
action = GrabSim_pb2.Action.ActionType.WalkTo,
values = [ -650.0, -1550.0,-15.0,-1,10]
))
Generate an object in the scene. You can add object in the specified position you need.
Request: ObjectList
Response: Scene - updated scene with object
Usage:
obj_list = [GrabSim_pb2.ObjectList.Object(X=25, Y=2, Yaw=15, Z=100, type=0)]
scene = stub.MakeObjects(GrabSim_pb2.ObjectList(objects=obj_list, scene=4))
Create an item of type “ADMilk” at the coordinates (X=30, Y=-260, Z=84)
scene = stub.GenerateObject(GrabSim_pb2.Object(name = "AA",type =
6,location = GrabSim_pb2_pb2.Location(X=30,Y=-260,Z=84)))
Query the model category of all walkers.
Request: Nothing
Response: model category of all walkers.
Usage:
print(stub.AcquireWalkers(GrabSim_pb2.NUL()))
All walker model categories to the appendix for details. We provide 52 categories of walker models with different appearances.
Add walkers to the scene.
Request: WalkerList - list of walkers to add
Response: Scene - updated scene with walkers
Usage:
updated_scene = stub.AddWalker(GrabSim_pb2.WalkerList(walkers=[walker1, walker2]))
Add 4 walkers at specified positions to the scene and update the scene:
scene = stub.Observe(GrabSim_pb2.SceneID(value=0))
walker_loc = [[120, -500], [-35, -385], [115, -360], [50,-392]]
walker_list = []
for i in range(len(walker_loc)):
loc = walker_loc[i]
action = GrabSim_pb2.Action(scene=0, action=GrabSim_pb2.Action.ActionType.WalkTo,
values=[loc[0], loc[1], 0, 0, 0])
scene = sim_client.Do(action)
print(scene.info)
walker_list.append(GrabSim_pb2.WalkerList.Walker(id=i, pose=GrabSim_pb2.Pose(X=loc[0], Y=loc[1], Yaw=90)))
scene = stub.AddWalker(GrabSim_pb2.WalkerList(walkers=walker_list, scene=0))
Control walker movements.
Request: WalkerControls - controls for each walker
Response: Scene - updated scene with walker positions
Usage:
updated_scene = stub.ControlWalkers(GrabSim_pb2.WalkerControls(controls=[control1, control2]))
Designate 4 walkers to go to the designated location respectively, using ControlWalkers will formulate the route and move according to the designated pose and speed.
scene = stub.Observe(GrabSim_pb2.SceneID(value=0))
walker_loc = [[95, 140], [93, -356], [123, 400], [97,-381]]
controls = []
for i in range(len(scene.walkers)):
loc = walker_loc[i]
is_autowalk = True
pose = GrabSim_pb2.Pose(X=loc[0], Y=loc[1], Yaw=180)
controls.append(GrabSim_pb2.WalkerControls.WControl(id=i, autowalk=is_autowalk, speed=200, pose=pose))
scene = stub.ControlWalkers(GrabSim_pb2.WalkerControls(controls=controls, scene=0))
Remove walkers from the scene.
Request: RemoveList - IDs of walkers to remove
Response: Scene - updated scene without removed walkers / delete specific walkers.
Usage:
updated_scene = stub.RemoveWalkers(GrabSim_pb2.RemoveList(walker_ids=[1, 2]))
Remove the walker and update the scene:
scene = stub.Observe(GrabSim_pb2.SceneID(value=scene_id)) # print(scene.walkers) scene = stub.RemoveWalkers(GrabSim_pb2.RemoveList(IDs=[1, 3], scene=scene_id))
Remove all walkers from the scene.
Request: SceneID
Response: Scene - updated scene without walkers / delete all walkers.
Usage:
update_scene = stub.CleanWalkers(GrabSim_pb2.SceneID(value=0))
Receive commands from the grpc server through BindCommand binding, such as clicking the record button in the VR scene, you will receive the “record” command
Request: SceneID
Response: Nothing
Usage:
for cmd in stub.BindCommand(GrabSim_pb2.SceneID(value=0)):
print(cmd)
from google.protobuf import message
import grpc
import numpy as np
import cv2
import matplotlib.pyplot as plt
import GrabSim_pb2_grpc
import GrabSim_pb2
channel = grpc.insecure_channel('localhost:30001',options=[
('grpc.max_send_message_length', 1024*1024*1024),
('grpc.max_receive_message_length', 1024*1024*1024)
]) # define the channel
stub = GrabSim_pb2_grpc.GrabSimStub(channel) # create a client
initworld = stub.SetWorld(GrabSim_pb2.BatchMap(count=4, mapID=3)) # init the world and creat 4 scenes(Caffe)
for i in range(len(initworld.scenes)):
scene=initworld.scenes[i] # choose the world's scenes[i] and print agent's location in the scene
print("scene %d, ginger location (%d,%d) direction %d"%(i,scene.location.X,scene.location.Y,scene.rotation.Yaw))
for j in range(len(initworld.scenes[i].objects)): # get the object and location in the scene
object = scene.objects[j]
print("scene %d, object %d: name %s, location (%d,%d,%d)"%(i,j,object.name,object.location.X,object.location.Y,object.location.Z))
scene = stub.Do(GrabSim_pb2.Action(scene=i, action=GrabSim_pb2.Action.WalkTo, values=[-500, 100, 90, -1, 10]))
# the agent naviagtion to (-500, 100, 90) and update scene.
print("scene %d, ginger moved to location %d,%d direction %d"%(i,scene.location.X,scene.location.Y,scene.rotation.Yaw))
message = stub.Capture(GrabSim_pb2.CameraList(scene=0, cameras=[GrabSim_pb2.CameraName.Head_Depth, GrabSim_pb2.CameraName.Head_Color])) # use the carmer in agent's head to get observation(depth, RGB, segmentation)
images = message.images
depth = np.frombuffer(images[0].data, dtype=images[0].dtype).reshape(
(images[0].height, images[0].width, images[0].channels))
rgb = np.frombuffer(images[1].data, dtype=images[1].dtype).reshape(
(images[1].height, images[1].width, images[1].channels))
seg = np.frombuffer(images[0].data, dtype=images[0].dtype).reshape(
(images[0].height, images[0].width, images[0].channels))
items = message.info.split(';')
seg_object_names = {}
for item in items:
key, value = item.split(':')
seg_object_names[int(key)] = value
# convert to BGR format
rgb = rgb[:, :, [2, 1, 0]]
scene = stub.Reset(GrabSim_pb2.ResetParams(scene=0)) # reset scene[0] in the world
import grpc
import GrabSim_pb2_grpc
import GrabSim_pb2
channel = grpc.insecure_channel('localhost:30001',options=[
('grpc.max_send_message_length', 1024*1024*1024),
('grpc.max_receive_message_length', 1024*1024*1024)
]) # define the channel
stub = GrabSim_pb2_grpc.GrabSimStub(channel) # create a client
initworld = stub.SetWorld(GrabSim_pb2.BatchMap(count=1, mapID=3)) # set world
scene = initworld.scenes[0] # choose scene
obj_list = [GrabSim_pb2.ObjectList.Object(X=25, Y=2, Yaw=15, Z=100, type="Mug")]
scene = stub.MakeObjects(GrabSim_pb2.ObjectList(objects=obj_list, scene=0))
objects = scene.objects
import grpc
import GrabSim_pb2_grpc
import GrabSim_pb2
channel = grpc.insecure_channel('localhost:30001',options=[
('grpc.max_send_message_length', 1024*1024*1024),
('grpc.max_receive_message_length', 1024*1024*1024)
]) # define the channel
stub = GrabSim_pb2_grpc.GrabSimStub(channel) # create a client
initworld = stub.SetWorld(GrabSim_pb2.BatchMap(count=1, mapID=3)) # set world
scene = stub.Observe(GrabSim_pb2.SceneID(value=0))
# acquire walkers
print(stub.AcquireWalkers(GrabSim_pb2.NUL()))
# add walkers
walker_loc = [[120, -500], [-35, -385], [115, -360], [50,-392]]
walker_list = []
for i in range(len(walker_loc)):
loc = walker_loc[i]
action = GrabSim_pb2.Action(scene=0, action=GrabSim_pb2.Action.ActionType.WalkTo,
values=[loc[0], loc[1], 0, 0, 0])
scene = sim_client.Do(action)
print(scene.info)
walker_list.append(GrabSim_pb2.WalkerList.Walker(id=i, pose=GrabSim_pb2.Pose(X=loc[0], Y=loc[1], Yaw=90)))
scene = stub.AddWalker(GrabSim_pb2.WalkerList(walkers=walker_list, scene=0))
# control walkers
walker_loc = [[95, 140], [93, -356], [123, 400], [97,-381]]
controls = []
for i in range(len(scene.walkers)):
loc = walker_loc[i]
is_autowalk = True
pose = GrabSim_pb2.Pose(X=loc[0], Y=loc[1], Yaw=180)
controls.append(GrabSim_pb2.WalkerControls.WControl(id=i, autowalk=is_autowalk, speed=200, pose=pose))
scene = stub.ControlWalkers(GrabSim_pb2.WalkerControls(controls=controls, scene=0))
# remove walkers
print(scene.walkers)
scene = stub.RemoveWalkers(GrabSim_pb2.RemoveList(IDs=[1, 3], scene=0))
print(scene.walkers)
# clean walkers
scene = stub.CleanWalkers(GrabSim_pb2.SceneID(value=0))
The walker’s start location and target location must be reachable. Otherwise the the scene.info will be unreachable. You can test whether the target location is reachable by the following method:
msg = stub.Do(GrabSim_pb2.Action( action = GrabSim_pb2.Action.WalkTo, values = [-2150,-1350,-100, 0, 10] # 0: Query only, not move # -1: Teleport to target position # 1: Navigation to target position )) print(msg.info)
Item |
---|
apple |
Cake |
Drinks |
Glass |
Saucer |
Door |
Knife |
Machine |
Bread |
Mug |
Packaged Coffee |
Spoon |
Cube Sugar |
Tray |
Straw |
Drink |
Take-Away Cup |
Tongs |
Vacuum |
Trash Bin |
Item |
---|
Room-Elevator |
Room-Dining |
Room-Bar |
Room-Drinking |
Room-Lobby |
Cake |
Plate |
Drinking Machine |
Bread |
Red Bull Drink |
Alcoholic Drink |
Kettle |
Fork |
Knife |
Alcoholic Drink (Bulk) |
Trolley |
Table |
Chair |
Teapot |
Glass |
Teacup |
Dixie Cup |
Tongs |
Item |
---|
Monitor |
Curtain |
Knife |
Pot |
Disc |
Plants |
Elevator |
Elevator Panel |
Trash Bin |
Door |
Chair |
Desk |
Chess |
WheelChair |
Gate |
Case |
Front Desk |
Closet |
Sofa |
TV |
Books |
Medicine |
Armrest |
Table |
Cloth |
Fridge |
Plant |
Teapot |
Microwave |
EmergencyKit |
Bed |
Fruit |
Type | ||||||
---|---|---|---|---|---|---|
walker: “Boy01” | walker: “Boy02” | walker: “Boy03” | ||||
walker: “Boy Euro01” | ||||||
walker: “ Girl01” | walker: “ Girl02” | walker: “Girl03” | ||||
walker: “Girl Euro01” | ||||||
walker: “ Male01” | walker: “ Male02” | walker: “Male03” | ||||
walker: “Male Afro01” | walker: “Male Afro02” | |||||
walker: “MaleAf roOw01” | ||||||
walker: “Male Amer01” | walker: “Male Amer02” | walker: “Mal eAmer03” | w alker: “MaleA mer04” | |||
walker: “Male Asia01” | walker: “Male Asia02” | |||||
walker: “MaleAs iaOw01” | walker: “MaleAs iaOw02” | walker: “MaleAsi aOwOw03” | ||||
walker: “Male Euro01” | walker: “Male Euro02” | |||||
walker: “MaleEu roOw01” | walker: “MaleEu roOw02” | |||||
walker: “Fe male01” | walker: “Fe male02” | walker: “F emale03” | w alker: “Fem ale04” | walke r:“Fem ale05” | walke r:“Fem ale06” | |
walker: “Female Afro01” | walker: “Female Afro02” | walke r:“Femal eAfro03” | w alker: “F emaleA fro04” | w alker: “F emaleA fro05” | w alker: “F emaleA fro06” | w alker: “F emaleA fro07” |
walker: “F emaleAf roOw01” | walker: “F emaleAf roOw02” | walker: “FemaleA froOw03” | ||||
walker: “Female Asia01” | walker: “Female Asia02” | walker: “Femal eEuro01” | w alker: “F emaleE uro02” | |||
walker: “F emaleEu roOw01” | walker: “F emaleEu roOw02” | walker: “FemaleE uroOw03” |
ID | Name |
---|---|
0 | Mug |
1 | Banana |
2 | Toothpaste |
3 | Bread |
4 | Softdrink |
5 | Yogurt |
6 | ADMilk |
7 | VacuumCup |
8 | Bernachon |
9 | BottledDrink |
10 | PencilVase |
11 | Teacup |
12 | Caddy |
13 | Dictionary |
14 | Cake |
15 | Date |
16 | Stapler |
17 | LunchBox |
18 | Bracelet |
19 | MilkDrink |
20 | CocountWater |
21 | Walnut |
22 | HamSausage |
23 | GlueStick |
24 | AdhensiveTape |
25 | Calculator |
26 | Chess |
27 | Orange |
28 | Glass |
29 | Washbowl |
30 | Durian |
31 | Gum |
32 | Towl |
33 | OrangeJuice |
34 | Cardcase |
35 | RubikCube |
36 | StickyNotes |
37 | NFCJuice |
38 | SpringWater |
39 | Apple |
40 | Coffee |
41 | Gauze |
42 | Mangosteen |
43 | SesameSeedCake |
44 | Glove |
45 | Mouse |
46 | Kettle |
47 | Atomize |
48 | Chips |
49 | SpongeGourd |
50 | Garlic |
51 | Potato |
52 | Tray |
53 | Hemomanometer |
54 | TennisBall |
55 | ToyDog |
56 | ToyBear |
57 | TeaTray |
58 | Sock |
59 | Scarf |
60 | ToiletPaper |
61 | Milk |
62 | Soap |
63 | Novel |
64 | Watermelon |
65 | Tomato |
66 | CleansingFoam |
67 | CocountMilk |
68 | SugarlessGum |
69 | MedicalAdhensiveTape |
70 | SourMilkDrink |
71 | PaperCup |
72 | Tissue |
Action.values param | Name |
---|---|
0 | Knee_X_Anchorn |
1 | Back_Z_Anchorn |
2 | Back_X_Anchorn |
3 | Back_Y_Anchorn |
4 | Neck_Z_Anchorn |
5 | Neck_X_Anchorn |
6 | Head_Y_Anchorn |
7 | LShlouder_X_Anchorn |
8 | LShlouder_Y_Anchorn |
9 | LElbow_Z_Anchorn |
10 | LElbow_X_Anchorn |
11 | LWrist_Z_Anchorn |
12 | LWrist_X_Anchorn |
13 | LWrist_Y_Anchorn |
14 | RShlouder_X_Anchorn |
15 | RShlouder_Y_Anchorn |
16 | RElbow_Z_Anchorn |
17 | RElbow_X_Anchorn |
18 | RWrist_Z_Anchorn |
19 | RWrist_X_Anchorn |
20 | RWrist_Y_Anchorn |
[1] | https://docs.unrealengine.com/5.0/en-US/quick-start-guide-for-using-container-images-in-unreal-engine/ |