Initial commit

.gitignore

@@ -0,0 +1,3 @@
+build
+*~
+*.swp

.gitmodules

@@ -0,0 +1,6 @@
+[submodule "3rdparty/eigen-git-mirror"]
+	path = 3rdparty/eigen
+	url = git@github.com:eigenteam/eigen-git-mirror.git
+[submodule "3rdparty/Pangolin"]
+	path = 3rdparty/Pangolin
+	url = git@github.com:stevenlovegrove/Pangolin.git

CMakeLists.txt

@@ -0,0 +1,17 @@
+# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
+cmake_minimum_required(VERSION 2.6.0)
+
+project(Replica)
+
+set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_CURRENT_LIST_DIR}/cmake/")
+
+set(THIRD_PARTY "${CMAKE_CURRENT_LIST_DIR}/3rdparty/")
+
+set(Eigen_INCLUDE_DIRS "${THIRD_PARTY}/eigen") 
+set(Pangolin_INCLUDE_DIRS "${THIRD_PARTY}/Pangolin/build/src/include")
+list(APPEND Pangolin_INCLUDE_DIRS "${THIRD_PARTY}/Pangolin/include") 
+set(Pangolin_LIBRARIES "${THIRD_PARTY}/Pangolin/build/src/libpangolin.so") 
+
+find_package(dl REQUIRED)
+
+add_subdirectory("./ReplicaSDK")

CODE_OF_CONDUCT.md

@@ -0,0 +1,2 @@
+# Code of Conduct
+Facebook has adopted a Code of Conduct that we expect project participants to adhere to. Please [read the full text](https://code.fb.com/codeofconduct) so that you can understand what actions will and will not be tolerated.

CONTRIBUTING.md

@@ -0,0 +1,31 @@
+# Contributing to Replica-Dataset
+We want to make contributing to this project as easy and transparent as
+possible.
+
+## Pull Requests
+We actively welcome your pull requests.
+
+1. Fork the repo and create your branch from `master`.
+2. If you've added code that should be tested, add tests.
+3. If you've changed APIs, update the documentation.
+4. Ensure the test suite passes.
+5. Make sure your code lints.
+6. If you haven't already, complete the Contributor License Agreement ("CLA").
+
+## Contributor License Agreement ("CLA")
+In order to accept your pull request, we need you to submit a CLA. You only need
+to do this once to work on any of Facebook's open source projects.
+
+Complete your CLA here: <https://code.facebook.com/cla>
+
+## Issues
+We use GitHub issues to track public bugs. Please ensure your description is
+clear and has sufficient instructions to be able to reproduce the issue.
+
+Facebook has a [bounty program](https://www.facebook.com/whitehat/) for the safe
+disclosure of security bugs. In those cases, please go through the process
+outlined on that page and do not file a public issue.
+
+## License
+By contributing to Replica-Dataset, you agree that your contributions will be licensed
+under the LICENSE file in the root directory of this source tree.

LICENSE

@@ -0,0 +1,36 @@
+Replica Dataset Research Terms 
+
+Before Facebook Technologies, LLC (“FB”) is able to offer you (“Researcher” or
+“You”) access to the Replica Dataset (the “Dataset”), please read the following
+agreement (“Agreement”). 
+
+By accessing, and in exchange for receiving permission to access, the Dataset,
+Researcher hereby agrees to the following terms and conditions:
+1.  Researcher may use, modify, improve and/or publish the Dataset only in
+connection with a research or educational purpose that is non-commercial or
+not-for-profit in nature, and not for any other purpose.
+2.  Researcher may provide research associates and colleagues with access to the
+Dataset provided that they first agree to be bound by these terms and
+conditions.
+3.  Researcher may use the Dataset in the scope of their employment at a
+for-profit or commercial entity provided that Researcher complies with Section 1
+of this Agreement. If Researcher is employed by a for-profit or commercial
+entity, Researcher's employer shall also be bound by these terms and conditions,
+and Researcher hereby represents that they are fully authorized to enter into
+this agreement on behalf of such employer.
+4.  THE DATASET IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL FB OR ANY
+CONTRIBUTOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE DATASET OR THE USE OR OTHER DEALINGS IN THE DATASET.
+5.  The law of the State of California shall apply to all disputes related to
+this Dataset.
+
+FAQ: 
+Q: I work for a commercial research organization. Can I use this data?
+A: Yes! We intend for employees of commercial research organizations to use
+this data as long as the purpose in using the data is for a research or
+educational purpose.
+
+

README.md

@@ -0,0 +1,138 @@
+# Replica Dataset
+
+The Replica Dataset is a dataset of high quality reconstructions of a
+variety of indoor spaces. Each reconstruction has clean dense geometry, high
+resolution and high dynamic range textures, glass and mirror surface
+information, planar segmentation as well as
+semantic class and instance segmentation.
+See the [technical report](https://arxiv.org/abs/1906.05797) for more details.
+
+![Replica Modalities](./assets/ReplicaModalities.png)
+
+The Replica SDK contained in this repository allows visual inspection of the
+datasets via the ReplicaViewer and gives an example of how to render out images
+from the scenes headlessly via the ReplicaRenderer. 
+
+For machine learning purposes each dataset also contains an export to the format
+employed by [AI Habitat](https://www.aihabitat.org/) and is therefore usable
+seamlessly in that framework for AI agent training and other ML tasks.
+
+## Citing the Replica Dataset
+
+If you use the Replica dataset in your research, please cite the following
+[technical report](https://arxiv.org/abs/1906.05797):
+```
+@article{replica19arxiv,
+  title =   {The {R}eplica Dataset: A Digital Replica of Indoor Spaces},
+  author =  {Julian Straub and Thomas Whelan and Lingni Ma and Yufan Chen and Erik Wijmans and Simon Green and Jakob J. Engel and Raul Mur-Artal and Carl Ren and Shobhit Verma and Anton Clarkson and Mingfei Yan and Brian Budge and Yajie Yan and Xiaqing Pan and June Yon and Yuyang Zou and Kimberly Leon and Nigel Carter and Jesus Briales and  Tyler Gillingham and  Elias Mueggler and Luis Pesqueira and Manolis Savva and Dhruv Batra and Hauke M. Strasdat and Renzo De Nardi and Michael Goesele and Steven Lovegrove and Richard Newcombe },
+  journal = {arXiv preprint arXiv:1906.05797},
+  year =    {2019}
+}
+```
+
+## Replica Dataset
+
+To download and decompress the dataset use the `download.sh` script:
+```
+./download.sh /path/to/replica_v1
+```
+
+The following 18 scenes are included in this initial release:
+
+![Replica Dataset](./assets/ReplicaDataset.png)
+
+![Replica Dataset](./assets/ReplicaDatasetFRL.png)
+
+Each Replica contains the following assets:
+```
+├── glass.sur
+├── habitat
+    ├── mesh_semantic.ply 
+    ├── mesh_semantic.navmesh
+    ├── info_semantic.json
+    ├── mesh_preseg_semantic.ply 
+    ├── mesh_preseg_semantic.navmesh
+    └── info_preseg_semantic.json 
+├── mesh.ply
+├── preseg.bin
+├── preseg.json
+├── semantic.bin
+├── semantic.json
+└── textures
+    ├── 0-color-ptex.hdr
+    ├── 0-color-ptex.w
+    ├── 1-color-ptex.hdr
+    ├── 1-color-ptex.w
+    ├── ...
+    └── parameters.json
+```
+The different files contain the following:
+- `glass.sur`: parameterization of glass and mirror surfaces.
+- `mesh.ply`: the quad mesh of the scene with vertex colors.
+- `preseg.json` and `preseg.bin`: the presegmentation in terms of planes and non-planes of the scene.
+- `semantic.json` and `semantic.bin`: the semantic segmentation of the scene.
+- `textures`: the high resolution and high dynamic range textures of the scene.
+- `habitat/mesh*semantic.ply`: the quad meshes including semantic or presegmentation information for AI Habitat. 
+- `habitat/info*semantic.json`: mapping from instance IDs in the respective `mesh_*.ply` to semantic names.
+- `habitat/mesh*semantic.navmesh`: navigation grid for AI Habitat.
+
+
+## Replica SDK
+
+### Setup
+The Replica SDK can be compiled using the build script via
+```
+git submodule update --init
+./build.sh
+```
+It requires the dependencies of
+[Pangolin](https://github.com/stevenlovegrove/Pangolin) and
+[Eigen](https://github.com/eigenteam/eigen-git-mirror)
+to be installed.
+
+### ReplicaViewer
+
+ReplicaViewer is an interactive UI to explore the Replica Dataset. 
+
+```
+./build/bin/ReplicaViewer mesh.ply /path/to/atlases [mirrorFile]
+```
+
+![ReplicaViewer](./assets/ReplicaViewer.png)
+
+The exposure value for rendering from the HDR textures can be adjusted on the
+top left. 
+
+### ReplicaRenderer
+
+The ReplicaRenderer shows how to render out images from a Replica for a
+programmatically defined trajectory without UI. This executable can be run
+headless on a server if so desired. 
+
+```
+./build/bin/ReplicaRenderer mesh.ply textures glass.sur
+```
+
+## Replica and AI Habitat
+
+To use Replica within AI Habitat checkout the AI Habitat Sim at [https://github.com/facebookresearch/habitat-sim](https://github.com/facebookresearch/habitat-sim).
+After building the project you can launch the test viewer to verify that everything works:
+```
+./build/viewer /PATH/TO/REPLICA/apartment_0/habitat/mesh_semantic.ply
+```
+
+## Team 
+
+Julian Straub,  Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J. Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, Anton Clarkson, Mingfei Yan, Brian Budge, Yajie Yan, Xiaqing Pan, June Yon, Yuyang Zou, Kimberly Leon, Nigel Carter, Jesus Briales,  Tyler Gillingham Elias Mueggler, Luis Pesqueira, Manolis Savva, Dhruv Batra, Hauke M. Strasdat, Renzo De Nardi, Michael Goesele, Steven Lovegrove, and Richard Newcombe.
+
+## Contact
+
+[Julian.Straub@oculus.com](Julian.Straub@oculus.com)
+
+## Acknowledgements
+
+The Replica dataset would not have been possible without the hard work and contributions of Matthew Banks, Christopher Dotson, Rashad Barber, Justin Blosch, Ethan Henderson, Kelley Greene, Michael Thot, Matthew Winterscheid, Robert Johnston, Abhijit Kulkarni, Robert Meeker, Jamie Palacios, Tony Phan, Tim Petrvalsky, Sayed Farhad Sadat, Manuel Santana, Suruj Singh, Swati Agrawal, and Hannah Woolums.
+
+## License
+
+See the LICENSE file for details.

ReplicaSDK/CMakeLists.txt

@@ -0,0 +1,58 @@
+# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
+cmake_minimum_required(VERSION 2.6.0)
+
+project(ReplicaSDK)
+
+set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_CURRENT_LIST_DIR}/../cmake")
+
+set(PTEX_SHADER_DIR "${CMAKE_CURRENT_LIST_DIR}/shaders" CACHE PATH "Where the shaders live")
+
+file(GLOB srcs_ptex ptex/*.cpp)
+file(GLOB hdrs include/*.h)
+
+set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -O3 -msse2 -msse3 -Wall -std=c++11 -fopenmp -DSHADER_DIR=${PTEX_SHADER_DIR}")
+
+add_library(ptex SHARED
+            ${srcs_ptex}
+            ${hdrs}
+)
+
+target_link_libraries(ptex
+                      ${Eigen_LIBRARIES}
+                      ${Pangolin_LIBRARIES}
+                      ${dl_LIBRARIES}
+                      stdc++fs
+)
+target_include_directories(ptex PUBLIC
+            ${Pangolin_INCLUDE_DIRS}
+            ${Eigen_INCLUDE_DIRS}
+            ${dl_INCLUDE_DIRS}
+            ${CMAKE_CURRENT_LIST_DIR}
+)
+
+include_directories(${Pangolin_INCLUDE_DIRS})
+include_directories(${EIGEN3_INCLUDE_DIR})
+include_directories(${dl_INCLUDE_DIRS})
+include_directories("./include")
+
+add_executable(ReplicaViewer src/viewer.cpp src/MirrorSurface.cpp)
+
+target_link_libraries(ReplicaViewer
+                      ${Pangolin_LIBRARIES}
+                      ${dl_LIBRARIES}
+                      GL
+                      GLEW
+                      ptex
+                      stdc++fs
+)
+
+add_executable(ReplicaRenderer src/render.cpp src/MirrorSurface.cpp)
+
+target_link_libraries(ReplicaRenderer
+                      ${Pangolin_LIBRARIES}
+                      ${dl_LIBRARIES}
+                      GL
+                      GLEW
+                      ptex
+                      stdc++fs
+)

ReplicaSDK/include/Assert.h

@@ -0,0 +1,13 @@
+// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
+#pragma once
+
+#include <iostream>
+
+#define ASSERT(x, ...)                                                                       \
+  do {                                                                                       \
+    if (!(x)) {                                                                              \
+      std::cout << "Assert failed: " #x << ", " << __FILE__ << ":" << __LINE__ << std::endl; \
+      std::cout << #__VA_ARGS__ << std::endl;                                                \
+      exit(-1);                                                                              \
+    }                                                                                        \
+  } while (false)

ReplicaSDK/include/EGL.h

@@ -0,0 +1,34 @@
+// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
+#pragma once
+
+#include <string>
+
+class EGLCtx {
+ public:
+  EGLCtx(const bool createCtx = true, const int cudaDevice = 0, const bool createSurface = true);
+  ~EGLCtx();
+  EGLCtx(const EGLCtx&) = delete;
+  EGLCtx& operator=(const EGLCtx&) = delete;
+
+  void* (*eglGetCurrentContext)(void);
+
+  void PrintInformation();
+
+ private:
+  void* display;
+  void* surface;
+  void* context;
+  void* handle;
+
+  const std::string lib;
+  const bool createdCtx;
+
+  unsigned int (*eglInitialize)(void*, int32_t*, int32_t*);
+  unsigned int (*eglChooseConfig)(void*, const int32_t*, void**, int32_t, int32_t*);
+  void (*(*eglGetProcAddress)(const char*))();
+  void* (*eglCreatePbufferSurface)(void*, void*, const int32_t*);
+  unsigned int (*eglBindAPI)(unsigned int);
+  void* (*eglCreateContext)(void*, void*, void*, const int32_t*);
+  unsigned int (*eglMakeCurrent)(void*, void*, void*, void*);
+  unsigned int (*eglTerminate)(void*);
+};

ReplicaSDK/include/MeshData.h

@@ -0,0 +1,43 @@
+// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
+#pragma once
+
+#include <pangolin/image/managed_image.h>
+#include <Eigen/Core>
+
+struct MeshData {
+  MeshData(size_t polygonStride = 3) : polygonStride(polygonStride) {}
+
+  MeshData(const MeshData& other) {
+    if (other.vbo.IsValid())
+      vbo.CopyFrom(other.vbo);
+
+    if (other.ibo.IsValid())
+      ibo.CopyFrom(other.ibo);
+
+    if (other.nbo.IsValid())
+      nbo.CopyFrom(other.nbo);
+
+    if (other.cbo.IsValid())
+      cbo.CopyFrom(other.cbo);
+
+    polygonStride = other.polygonStride;
+  }
+
+  MeshData(MeshData&& other) {
+    *this = std::move(other);
+  }
+
+  void operator=(MeshData&& other) {
+    vbo = (std::move(other.vbo));
+    ibo = (std::move(other.ibo));
+    nbo = (std::move(other.nbo));
+    cbo = (std::move(other.cbo));
+    polygonStride = other.polygonStride;
+  }
+
+  pangolin::ManagedImage<Eigen::Vector4f> vbo;
+  pangolin::ManagedImage<uint32_t> ibo;
+  pangolin::ManagedImage<Eigen::Vector4f> nbo;
+  pangolin::ManagedImage<Eigen::Matrix<unsigned char, 4, 1>> cbo;
+  size_t polygonStride;
+};