Hands on: Representing plants as Nerf

• Public data sets
  • Datasets for agriculture/forestry (mostly photo-based for ag.):
  • GitHub
• Nerfs from flowers
  • Tree and drones
  • Tree capture challenges
• Tests
  • Drone flight free standing leavy tree, Orbits in 4 heights
    • Mesh
    • Footage: High Orbit
    • Mid
    • Low

Table of contents generated with markdown-toc

Public data sets

Datasets for agriculture/forestry (mostly photo-based for ag.):

• https://github.com/blutjens/awesome-forests GitHub - blutjens/awesome-forests: 🌳 A curated list of ground-truth...

• https://plantcv.danforthcenter.org/pages/data.html

• https://www.quantitative-plant.org/dataset

Nerfs from flowers

• Good fit for frequency based encoding
• Easy to capture from ground

See also my macro photo test

Tree and drones

Tree capture challenges

• Wind moves leaves, so they look blurry
• Dense leaves hide branches, leaves tend to look blurry
• Shadows, High contrast, not easy to shot drone HDR

Resolution limitations, nerf dont enlarge things

• Colmap tracking: dont track flights separately
  • After frame crop: Single camera specified, but images have different dimensions
  • Register different flighs globally : Different orbit trackings dont align globally https://colmap.github.io/faq.html?highlight=model_merger#register-localize-new-images-into-an-existing-reconstruction

Lower and mid-low cameras

Tests

Synthetic Nerf data in NVIDIA instant nerf (from https://drive.google.com/drive/folders/1JDdLGDruGNXWnM1eqY1FNL9PlStjaKWi)

Render and mesh (inflation parameters dont work well for leaves)

Drone flight free standing leavy tree, Orbits in 4 heights

DJI Footage (4 flights, ca 50s each, 25 fps, ca 2K resolution). 4GB memory

2 layers, 11 GB memory

Codec:

Cutting planes

First test (Realtime rendering instant Nerf, fps 6, from 300 images, aabb 16): Details are blurry

Mesh

max 700 voxels on this machine

Footage: High Orbit

Mid

Low