[R] Using AI as a spatial software generator to create 3D objects that are inherently programmable

I'm one of the co-authors of this paper. It's a seminal work in exploring the properties of 3D generated by LLMs via spatial programming.

I've set up visual demonstrations of such 3D objects at: https://nova3d.xyz/

Scroll down and notice how the various 3D objects are all composed of logical parts and enable natural movements out of the box. There's a github repo in there as well.

Under the hood:
We found that 3D that exists as software is much more useful than typical monolithic mesh blobs generated by traditional AI 3D generators. For instance they are animation-ready and programmable from inception. They can contain the logic - at birth - to appear differently in weak compute environments (e.g. mobiles) vs powerful environments (e.g. sophisticated game engines). They can be built with full hierarchical structure and hinge/socket articulation at authoring time.

They lag behind traditional AI 3D generators in creating complex organic shapes. But it naturally feels like code will eventually eat all 3D, as LLMs are getting better and better at spatial coding. Industries most disrupted will be industrial design, game development, simulations and AR/VR/XR.