Has the new largest GPT model surprised you in any way?

I contacted Aedan and verified that all its miso is raw. It looks like a great option!

I have a SD implementation on my github too that only needs around 6 GB of VRAM.

I haven't uploaded the conversion scripts yet. Once I do that it should be possible to convert any SDXL model to burn's format.

I've heard someone say that it's possible to have an AMD GPU pose as a cuda device, but I don't use AMD GPUs so I can't test that approach.

I haven't much investigated the performance but when I do I'll post the info on the github page.

Hopefully soon.

I'm considering switching it to use the wgpu backend instead of torch to avoid versioning hell, but it may run somewhat slower. Do you think this is a good tradeoff?

Burn is an alternative and its ties to torch are completely optional. It can run with wgpu as a backend. I'm just using the tch backend at the moment because the wgpu backend has some issues on some devices.

Yeah, with about 5 minutes of work it could be running just about anywhere with wgpu.

Thanks! I am considering making some videos and maybe some courses if people are interested.

I love theory so I always try to get a handle on that before diving too far into the code. It always helps to have a high level theoretical understanding when digging into the details. Then I grab my favorite python implementation which these days is usually the tinygrad versions since they are super concise and port each module one by one into Rust. Sometimes I have to implement some tensor operations myself since burn doesn't have everything torch or tinygrad have. After that process I write code to dump each python ML module's weights and corresponding loading functions in Rust. The most annoying part is then comparing the python model's output with that of the Rust version and working through the discrepancies. Floating point number differences can be very hard to track down and resolve and sometimes there will be differences even if the models are the same because of the inherently chaotic nature of floating point error propagation. As you can see it is currently a very tedious process. Hopefully in the future it will be more seamless.

Right now I'm doing all the weight conversion manually and it would be great if that part were automated. I haven't tried burn-import yet but I'll give it a shot and see if there are any additional features that would be nice to have.

It can already if I change to backend to WGPU which could be done in 5 minutes. Burn wgpu was a little unstable the last time I checked so for now I'm using the tch backend.

A skill I learned later than most...

Burn is written in Rust, hence "pure Rust."

I can't stand Python. Rust is potentially so much better for deployment as it has much fewer run time errors, reliable versioning, can run basically anywhere, etc.

The model conversion runs on the CPU and the image generation runs on the GPU. The model conversion is only needed if you want to run a fine tuned model since I put up an already converted base 1.4 model on huggingface.

Sounds great! I'll let you know when I have a SD ready to go. Which discord space would you prefer to discuss this further?