Tesla AI day - deep dive into Teslas Neural Network

[LetterRip]

Those of you with Neural Net design background - Tesla do a really nice dive into the design of their network,

https://www.youtube.com/watch?v=j0z4FweCy4M

Those without a neural network background can still follow it - long video (skip the first 30 minutes or so which are simply waiting)

[LetterRip]

They also do a deep dive on their training architecture - sounds really impressive although I'm fairly ignorant on this topic (I understand hardware architecture but no idea how their design compares to state of the art).

Also Tesla is apparently going to do humanoid robotics, plan to have a prototype out in a year.

[LetterRip]

Unsatisfactory Q&A - Dojo sounds like it is far less complete than the presentation suggested - heat dissipation, bandwidth, and scheduling are all really important issues that it seems they are far from solving (According to one author the total dissipated power for a single one of their chip packages would apparently be more heat than a single rack could handle, let alone 12 of them in a rack as suggested by the presentation).

It sounds like most of the tasks they need to do though are fairly independent of each other (mostly video processing for training) so  the scheduling and internode bandwidth aren't really important for that.

[LetterRip]

So, people who know a great deal about chip design, packaging, and integration/scaling wrote up an article about the DOJO architecture, and they are extremely impressed.

Rolling it all up, cost equivalent versus Nvidia GPU, Tesla claims they can achieve 4x the performance, 1.3x higher performance per watt, and 5x smaller footprint. Tesla has a TCO advantage that is nearly an order magnitude better than an Nvidia AI solution. If their claims are true, Tesla has 1 upped everyone in the AI hardware and software field. I'm skeptical, but this is also a hardware geek's wet dream.

https://semianalysis.substack.com/p/tesla-dojo-unique-packaging-and-chip

[LetterRip]

This is a cooling solution for 8 500W NVidia A100 GPU's  (4000W) so not too different from a solution that would be needed for 12 400W D1's (4800W),

https://www.hpcwire.com/2021/06/28/inspur-releases-liquid-cooling-ai-server-which-supports-500w-a100-gpu-at-isc-high-performance-digital-2021/

So the 'difficulties cooling' claims seem likely over stated.

[TheDrake]

If their claims are true

Big caveat there. Since this is proprietary and they are not selling their solution outside the company, that's a hard one to verify.

As to whether the author of the article is bona fide and knows what he's talking about? Well he's not a tech guy and he was the front desk manager at a best western 7 years ago.

He uses as a reference one of his own articles:

We try to stay fact based with our analysis, but recently an image was posted of a Tesla chip that leads to an immense amount of speculation that we would like to dip out toes in.

I'm sure they have some cool tech, but I'd guess those claims are exaggerated at best and pure fantasy at worst.

[LetterRip]

If their claims are true

Big caveat there. Since this is proprietary and they are not selling their solution outside the company, that's a hard one to verify.

Sure, but I don't think that is unreasonable to think that a direct statement is being honest.  I think they could be mistaken in their belief, or overly optimistic in interpretation, but I doubt they are lying.

For the machine learning part Lex Fridman (MIT DeepLearning instructor, does computer vision and driving research) was deeply impressed,

https://www.youtube.com/watch?v=ABbDB6xri8o

As to the author good point, his analysis seems reasonable and expertise can be acquired through independent study but his formal training appears limited to an MIS.

[TheDrake]

Thanks, LR, I'll check that out. Might be worth watching through a video for that, considering that I work for an AI silicon company I can even do it on company time.