Image-blaster: Creates 3D environments, SFX, and meshes from a single image

If you haven't tried AI modeling pipelines in the last year you'll be surprised.

The star of the show here is https://platform.worldlabs.ai/ (author works there, I don't) which is really good. There's also Meshy.ai (which this repo doesn't seem to use?) for non-scene stuff that's right up there in quality. There's texturing, auto-rigging, etc.

The latest VLLM models have true pixel image grounding which means you can totally ask your AI about pixel coordinates of things, so you get 3d perception for edits and anything else you need.

I'm actually surprised I don't see this stuff being used more; I think it's because most pipelines are hard-baked with assumption that your 3D assets are files you get from an artist, not something you can imagine up in minutes in a script. The technology is moving faster than the industry can keep up with.

If you haven't tried AI modeling pipelines in the last year you'll be surprised.

The latest VLLM models have true pixel image grounding which means you can totally ask your AI about pixel coordinates of things, so you get 3d perception for edits and anything else you need.

This is cool as hell.

I remember like seventeen years years ago, Microsoft had "PhotoSynth", which would make 3D environments based on a bunch of images, and seventeen-year-old-tombert thought it was one of the most amazing things to ever be done on a computer.

Doing this with just one image makes this at least an order of magnitude cooler. I will be playing with this over the weekend.

I see it used worldlabs, i’ve tested it quite a bit and no results were not really that usable, it hallucinated so many parts outside of the wall that made no sense. He will be fine if hallucinated and it made sense but if it doesn’t make sense, I’m not sure what the point of inputting a single image is. I’ve actually found better luck using gpt image 2 instead.

I wonder if there is something similar but for creating isometric sprites? I burned through $30 yesterday realizing that I can't just get image gen to give me isometric static/animated sprites with consistency....even the best image gen cannot do this and im just baffled how difficult isometric sprite is compared to 3d mesh gen

I'm at a crossroad , do I opt for 3d mesh isometrics with more hardware requirements for mobile phones or stick to isometric sprite which nobody seems to be generating via AI reliably (happy to be corrected here if anybody does find a way)

My team is working in the character animation space which might complement this: https://uthana.com/

Example: https://uthana.com/app/preview/cXi2eAP19XwQ/mH7opbcqZE4P

Is this in the same vein as TRELLIS?

https://github.com/Microsoft/TRELLIS

I've been trying to use this to generate 3d character models from images. I am enjoying 3d printing these models to mess with my kids.

Not much of what I've found runs on local models but I'm always on the lookout. Meshy.ai (mentioned here) offers really nice generation but the cost adds up quickly.

So Blade Runner's Esper photo analysis went from ruining the suspension of disbelief to reality quicker then most magic.

Curious about the actual architecture. From the outside it looks like Gaussian splatting anchored to roughly one viewpoint, since the moment you wander outside the original frame or behind an object, it becomes messy. But Ben Mildenhall is one of the co-founders and a NeRF co-author (https://arxiv.org/abs/2003.08934), so I'm betting that whatever they're doing is more interesting than naive splatting. Curious if OP can share anything about the pipeline.

What about creating 3d meshes from multiple photos of the same object?

That's pretty wild. How does it handle different lighting conditions in the source image? Curious if the results look natural or if they need a lot of tweaking.

I’m ready to make a game with this, or something similar. Open to suggestions on tooling and asset pipelines that utilize AI, if anyone has any suggestions or guides.

Very cool.

May I ask if Claude is the only option to use the tool?

Sol Roth

> I'm actually surprised I don't see this stuff being used more;

There's very little incentive to publicly admit you're using this tech. In fact there are a lot of reasons not to.

What's the best option for converting house blueprints or 3D rendered images back to models?

Outside trained images or objects, Hunyuan falls apart pretty hard.

Out of 30 objects I tried, only 4 had relative success. And even then, the topo ain't great.

Homepage for World Labs mentioned in parent comment: https://www.worldlabs.ai/

> I'm actually surprised I don't see this stuff being used more;

There's very little incentive to publicly admit you're using this tech. In fact there are a lot of reasons not to.

That's pretty wild. How does it handle different lighting conditions in the source image? Curious if the results look natural or if they need a lot of tweaking.

I’m ready to make a game with this, or something similar. Open to suggestions on tooling and asset pipelines that utilize AI, if anyone has any suggestions or guides.

Very cool.

May I ask if Claude is the only option to use the tool?

Sol Roth

This is cool as hell.

Doing this with just one image makes this at least an order of magnitude cooler. I will be playing with this over the weekend.

yeah I remember that was pretty neat each generation 3d stuff gets wilder and wilder

I used to spend all day on Bryce3D creating 3d landscapes, leaving computer on fall night to render like 10 seconds of video of a flyover sunset

bit of a rant here but we are definitely speedrunning 3d and its just going to get wilder once we get glass free bounded AR...projecting 3d video streams and objects in front of our phones (this one I know Samsung is already working on) and rooms

Photosynth was awesome, I really miss it, but it was more of a panorama tool than a 3d environment.

My pixel6 has a photo sphere mode on the camera which is the same thing

Ugh i miss that program

Yeah, even the latest version, Marble 1.1 which this repo uses can make a royal mess of things, especially outdoor environments.

You're better off creating a 3D model and then taking image-grabs of it facing different directions/doing different poses from an isometric perspective to get your isometric sprites, with the genAI tools of today without a skilled 2D artist at hand.

If you aren't ready to rig and adjust model poses in a 3D tool, you might be better off generating each movable model part as a separate mesh and just arranging them in space before doing the above.

> I'm at a crossroad , do I opt for 3d mesh isometrics with more hardware requirements for mobile phones or stick to isometric sprite which nobody seems to be generating via AI

Just find an artist or learn to draw

I had a similar experience recently while helping my 5 year old daughter vibe code a sandcastle-themed tower defense game (https://sandcastles.finley.lol).

I ended up thinking it might be easier to generate rigged models, animate them, and capture from an iso perspective, then do some kind of pixel art style transfer on the masked sprite sheet. Eventually I realized my kid didn't really care too much about the visuals so I didn't get too far with it.

My team is working in the character animation space which might complement this: https://uthana.com/

Example: https://uthana.com/app/preview/cXi2eAP19XwQ/mH7opbcqZE4P

That looks like just what I have been looking for. Now time to see how well it works!

Is this in the same vein as TRELLIS?

https://github.com/Microsoft/TRELLIS

I've been trying to use this to generate 3d character models from images. I am enjoying 3d printing these models to mess with my kids.

Not much of what I've found runs on local models but I'm always on the lookout. Meshy.ai (mentioned here) offers really nice generation but the cost adds up quickly.

There are quite a few, now, and more coming out regularly to surprisingly little fanfare.

Tencent's Hunyuan3D (https://github.com/Tencent-Hunyuan) is a single/multi view photogrammetry replacement, which image-blaster is based on.

Facebook Research has extended SAM to 3D (https://github.com/facebookresearch/sam-3d-objects), separating as 'Objects' and 'Body'.

The workflows to make meshes watertight for 3D printing are all pretty effective.

This is more like a Claude-based skill set that orchestrates a bunch of different, separate systems. The closest equivalent to Trellis would probably be its usage of Huyuan-3D, which it uses to create some of the 3D object models.

From what I can tell, it takes an image and first segments it into objects versus environment then sends the environment to Marble 1.1 to generate a Gaussian splat,sends all the isolated individual objects to Hunyuan to generate GLB model files.

So Blade Runner's Esper photo analysis went from ruining the suspension of disbelief to reality quicker then most magic.

Well, in blade runner he looks around a corner and zooms in microscopic detail on something not visible from the photo.

But the esper interface is all voice activated, and doesn't talk back - which I think is very prescient, and more likely the way things will go. I'd much rather voice assistants just did the thing that I want them to do rather than talk back to me

What about creating 3d meshes from multiple photos of the same object?

I've had some success with that. It's pretty neat!

Haven't used it professionally mainly because the titles I've worked on lately aren't realistic so you can't really procure the materials to scan.

Homepage for World Labs mentioned in parent comment: https://www.worldlabs.ai/

yeah I remember that was pretty neat each generation 3d stuff gets wilder and wilder

I used to spend all day on Bryce3D creating 3d landscapes, leaving computer on fall night to render like 10 seconds of video of a flyover sunset

Ugh i miss that program

Yeah, even the latest version, Marble 1.1 which this repo uses can make a royal mess of things, especially outdoor environments.

> I'm at a crossroad , do I opt for 3d mesh isometrics with more hardware requirements for mobile phones or stick to isometric sprite which nobody seems to be generating via AI

Just find an artist or learn to draw

That looks like just what I have been looking for. Now time to see how well it works!

There are quite a few, now, and more coming out regularly to surprisingly little fanfare.

Tencent's Hunyuan3D (https://github.com/Tencent-Hunyuan) is a single/multi view photogrammetry replacement, which image-blaster is based on.

Facebook Research has extended SAM to 3D (https://github.com/facebookresearch/sam-3d-objects), separating as 'Objects' and 'Body'.

The workflows to make meshes watertight for 3D printing are all pretty effective.

What's the best option for converting house blueprints or 3D rendered images back to models?

There are not good workflows for doing this for a whole scene, especially where accuracy needs to be preserved. One shot models seem to be very good at providing 3D from a single angle, but multishot is very shoddy at this point, and without seeing behind things you have no clue about what is actually there.

Also, try meshy and look at how many polygons or triangles you get from any of the model objects. Hundreds of thousands, when you retopologize still goes to the high tens of thousands.

Outside trained images or objects, Hunyuan falls apart pretty hard.

Out of 30 objects I tried, only 4 had relative success. And even then, the topo ain't great.

> Outside trained images or objects, Hunyuan falls apart pretty hard.

It's not too bad. It's fantastic for blocking out scenes for upscaling and conversion to video. Use it with 3D and a GPT Image 2 or Nano Banana Pro pass.

Eg.

https://getartcraft.com/media/m_xxtxfsdq3m5fwg5b8hjvkwxp4qyb...

https://getartcraft.com/media/m_gb4qexzqz1grskhn28f235fdtxfk...

https://getartcraft.com/media/m_p6wh750n0gc1s4a0ts88kp5pvnvn...

(All of ArtCraft is on github: https://github.com/storytold/artcraft )

> And even then, the topo ain't great.

This is true. The topo sucks.

Still, people are making delightful games like this (which was 100% vibe coded) :

https://tinyskies.vercel.app/

https://github.com/dannylimanseta/tinyskies

Photosynth was awesome, I really miss it, but it was more of a panorama tool than a 3d environment.

My pixel6 has a photo sphere mode on the camera which is the same thing

You could actually make it have a rough 3D environment as well. Their demo had a model of Piazza San Marco with dots to estimate the actual buildings and the like.

If you aren't ready to rig and adjust model poses in a 3D tool, you might be better off generating each movable model part as a separate mesh and just arranging them in space before doing the above.

yeah 3D comes up often but the economics don't work for the large scale I am trying to do.

I had a similar experience recently while helping my 5 year old daughter vibe code a sandcastle-themed tower defense game (https://sandcastles.finley.lol).

That's a cute looking game! I have considered using 3D mesh models but to generate a highly detailed, textured 3d mesh it still costs quite a bit especially when you need to do this at scale

Also, try meshy and look at how many polygons or triangles you get from any of the model objects. Hundreds of thousands, when you retopologize still goes to the high tens of thousands.

> Outside trained images or objects, Hunyuan falls apart pretty hard.

It's not too bad. It's fantastic for blocking out scenes for upscaling and conversion to video. Use it with 3D and a GPT Image 2 or Nano Banana Pro pass.

Eg.

https://getartcraft.com/media/m_xxtxfsdq3m5fwg5b8hjvkwxp4qyb...

https://getartcraft.com/media/m_gb4qexzqz1grskhn28f235fdtxfk...

https://getartcraft.com/media/m_p6wh750n0gc1s4a0ts88kp5pvnvn...

(All of ArtCraft is on github: https://github.com/storytold/artcraft )

> And even then, the topo ain't great.

This is true. The topo sucks.

Still, people are making delightful games like this (which was 100% vibe coded) :

https://tinyskies.vercel.app/

https://github.com/dannylimanseta/tinyskies

yeah 3D comes up often but the economics don't work for the large scale I am trying to do.

That's a cute looking game! I have considered using 3D mesh models but to generate a highly detailed, textured 3d mesh it still costs quite a bit especially when you need to do this at scale

You could actually make it have a rough 3D environment as well. Their demo had a model of Piazza San Marco with dots to estimate the actual buildings and the like.

Oh yes, I remember that now!

Well, in blade runner he looks around a corner and zooms in microscopic detail on something not visible from the photo.

I've had some success with that. It's pretty neat!

Haven't used it professionally mainly because the titles I've worked on lately aren't realistic so you can't really procure the materials to scan.

I've never forgotten this SIGGRAPH demo from almost twenty years ago, in which the authors effectively switch camera and light source computationally (... in a static scene) [1]

Ever since then, I have viewed scenes such as the "lingerie store scene" from Enemy of the State [2] with a little bit less eye rolling...

[1] - https://www.youtube.com/watch?v=p5_tpq5ejFQ

[2] - https://youtu.be/3EwZQddc3kY?t=6

any software specifically? I remember like 5 years ago or so seeing someone taking a photo with a DLSR of a chipped stair step and 3d printing an addition to it which filled the chipped part like a glove and it was looking like a lego brick.

I haven't found it though. Only some "Kiri Engine" which requires phone.

Oh yes, I remember that now!

`image-blaster`

Creates 3D environments, SFX, and meshes from a single image using Claude skills, World Labs, and FAL.

Can take you from an image to a fully meshed 3D environment in < 5 minutes, great for jumpstarting 3D work. Go full blast.

Quickstart

Open a Terminal, enter git clone https://github.com/neilsonnn/image-blaster
Enter the directory with cd image-blaster
Run claude (install with curl -fsSL https://claude.ai/install.sh | bash)
Say hello to Claude, and give them your API key for World Labs and FAL.
Put an image into input/ directory and ask Claude to blast it and confirm each step with me.

Description

By default image-blaster will use your input image to create:

3D models (.glb, .obj) of all dynamic objects
Gaussian splat (.spz) of the static environment,
Ambient looping sound and object specific physics SFX (.mp3)

Extensions

You can embed image-blaster under the assets of any game engine, DCC software, or web app.

Unity, Unreal, or Godot game engine
Blender, 3DS Max, or Maya or other DCC software
Three.js web app or Electron app

Advanced

IMAGE-BLASTER uses a few generation models:

marble-1.1 - World Labs Marble model creates the explorable environment.
nano-banana - default image edit preference for source cleanup, clean plates, and object reference images.
gpt-image-2 - alternate image edit provider when the edit skill is asked to prefer it.
hunyuan-3d - Hunyuan 3D model creates 3D object models through FAL.
elevenlabs-sfx - ElevenLabs sound effects model creates ambient and object-specific sounds.

3D model creation supports these Hunyuan parameters:

--face-count <40000-1500000>: target face count. IMAGE-BLASTER defaults to 50000; Hunyuan's API default is 500000.
--enable-pbr true|false: enable PBR material generation. Defaults to true.
--generate-type Normal|LowPoly|Geometry: Normal creates a textured model, LowPoly applies polygon reduction, and Geometry creates a white geometry-only model. Defaults to Normal.
--polygon-type triangle|quadrilateral: polygon type for LowPoly. Defaults to triangle.

Examples

Video game level concepts? IMAGE-BLAST it.
Your childhood bedroom? IMAGE-BLAST it.
Need an environment for a robot? IMAGE-BLAST it.
A film location scout? IMAGE-BLAST it.
An architectural rendering? IMAGE-BLAST it.

Development

remove /app from the .claudeignore file to give Claude the ability to change the React viewer.