Plan-E /Shap-E

1

Has anyone played around with these from OpenAI? I want to see how well they work but need help with working out how to install and run these.

2 Likes
2

This is the first I’ve heard of either one, but installation (found here) looks straightforward:

Install with pip install -e ..

To get started with examples, see the following notebooks:

  • sample_text_to_3d.ipynb - sample a 3D model, conditioned on a text prompt.
  • sample_image_to_3d.ipynb - sample a 3D model, conditioned on a synthetic view image. To get the best result, you should remove background from the input image.
  • encode_model.ipynb - loads a 3D model or a trimesh, creates a batch of multiview renders and a point cloud, encodes them into a latent, and renders it back. For this to work, install Blender version 3.3.1 or higher, and set the environment variable BLENDER_PATH to the path of the Blender executable.

You’ll need Jupyter (formerly known as IPython) to play around with those notebook files.

1 Like
3

From what I saw it has to be run in a Linux environment? is that what you found?

1 Like
4

I don’t see why it couldn’t be run in Windows or MacOS, but maybe I’ve missed something. Where did you read that?

1 Like
5

Thanks for sharing @BrettfromOz!
It runs on any OS, but there are lots of heavy dependencies you have to install (~20+ gigs worth). Also, if you don’t have a GPU, it is likely not worth the effort to install since you won’t be able to effectively generate results.
I spun up an instance of the “text to 3d” model in the cloud here if you want to play with it ( I ran a few examples towards the bottom). You can change the prompt to see new outputs. I didn’t try the “image to 3d” yet.

2 Likes
6

Thanks Chris…what would be a good baseline system to run this on? I’m so far out of the loop on computers nowadays lol. This is something I really want to master as it’s clearly the way things are going

···

On Wed, May 24, 2023, 1:35 PM Chris via MakeICT Forum <noreply@talk.makeict.org> wrote:

| chris_w MakeICT Member
May 24 |

  • | - |

Thanks for sharing @BrettfromOz!
It runs on any OS, but there are lots of heavy dependencies you have to install (~20+ gigs worth). Also, if you don’t have a GPU, it is likely not worth the effort to install since you won’t be able to effectively generate results.
I spun up an instance of the “text to 3d” model in the cloud here if you want to play with it ( I ran a few examples towards the bottom). You can change the prompt to see new outputs. I didn’t try the “image to 3d” yet.

Visit Topic or reply to this email to respond.

To unsubscribe from these emails, click here.

1 Like
7

As long as you have a GPU, you should be good, you don’t need a really expensive one. That said, you may consider using cloud provided GPUs instead, at least until you’ve decided that buying one yourself is worth it. More often than not, the free tier versions of the cloud services provide more than enough compute for running models. The notebook link I posted above uses a free configurable cloud GPU, and it takes just a couple minutes to output a 3d result.

1 Like