What is the most cost effective way for an individual researcher to purchase fast compute for deep learning tasks?

I program in MATLAB and want to substantially increase the computing speed for deep neural network projects. I am talking about a speed up factor in the neighborhood of 50 times my current speed. My current computer is a 2015 MacBook Pro 15" with 2.2 GHz Quad-Core Intel Core i7 and 16 GB 1600 MHz DDR3.

I cannot easily use cloud computing because my MATLAB license doesn't support it. I suppose the best way is to buy a Windows computer with a NVIDIA chip so MATLAB can use the GPUs.

What alternatives do I have? How much faster would a new MacBook Pro with a M3 Max chip be (I'm guessing less than 10x)? I could consider buying a desktop rather than a laptop if necessary. My budget can accommodate a fairly large investment (from the perspective of a hobbyist).

Answer 1

[John D'Errico]

No matter what, I would be looking towards a desktop machine. Then you get the advantage of adequate fans to keep those cores humming along. Without adequate cooling, your machine will throttle speeds down to avoid a meltdown.

It is 100% certain you will not get a 50x speed boost merely by going to a faster MacBook pro, M3 chip or not. I'd expect something on the order of a 3x speed boost that way, hardly a 10x boost. Even there, again, the lack of adequate fans will hamper you, as once you start to get all cores humming along, it will generate some serious heat, and your computer hates heat.

In the end, it does not matter how fast and brutish a machine you have, as John's law will apply. (A law I formulated perhaps 35-40 years ago.) It states that the size of computing projects always grows to be just a wee bit larger than the most powerful computer you can afford or find. This is beause the small problems are easy to solve, and someone else has probably already solved them anyway.

[Kevin Johnson]

John,

Thank you for replying. Yes, I've experienced John's law many, many times.

I'm looking for specific recommendations. Windows machine versus e.g. a Mac Pro desktop? How much faster can I expect a high-end Windows desktop to run? What specifically should I be ordering to accomplish this with parallel computing in MATLAB?

I'm hoping somebody here will have specific recommendations for the latest hardware out there.

Kevin

[John D'Errico]

I assume Walter will look in at some point, someone with experience on both sides of the fence. But in order to get a 50 times speed bump, you will need something serious.

For example, if I just run bench on my current imac, I get this:

Not terribly fast, but then it is a few years old now. Even so, it suggests that the fastest computer it might be compared to is not much more than twice the speed of mine.

As such, the only way to get a serious speed bump up will be to get something with seriously more cores. You are never going to afford to buy something with a 50x clock speed unless you work for the NSA anyway. So you need cores. Not just logical cores either. And even then, my machine already has 8 real cores. So even if you get 16 or 32 cores, all that will give you is a maximum of 4x.

This means you will need to get into the use of your GPU, and heavily so.

[Kevin Johnson]

I see. There is a consumer market out there for Windows desktop computers with many GPUs, for serious gamers. I suspect that will be the direction I will need to go. Even then, 50x is an ambitious target, perhaps extravagantly so. I don't have a feel for it. I don't know anything about Windows hardware these days.

My wife says getting a Windows computer would be against my DNA. :)

[John D'Errico]

I would not do it myself, even were it some guy named Darth telling me to come over to the dark side. Yes, I admit in my last few years consulting at Kodak, they insisted I use a Windoze laptop. I hated the thing. It was 20 years ago (hard to believe) but even so. Yes, they were paying me well, enough to get me to hold down the vomit.

Seriously, before you make a jump like that, you should contact tech support ot TMW. Ask them what it would take to get the kind of speed boost you want, and if it would even be possible on anything you could afford. I would want to be completely confident in my choice before I invest any kind of serious money, especially if it would require a change of environments.

[Kevin Johnson]

Yes, that's good advice, I'll see if they'll respond.

[Walter Roberson]

I have one of the last of the Intel iMac. Bench shows a relative speed of about 40. It also shows Mac Mini Apple M2 with a relative speed of about 80 -- a decent performance boost but not even close to a factor of 50.

If I understand correctly, NVIDIA performance is better on Linux than on Windows, for the same hardware.

[Joss Knight]

50x is completely possible. All you need to do is start with terrible code, then improve your code and get a GPU at the same time.

In spite of the M-series chips, Apple just aren't all that interested in high performance computing, which is why you won't see any data centers crammed with Apple hardware. You need external GPUs, which Apple doesn't like.

If you want beauty, go Mac. If you want to configure your hardware for high performance, go Linux. With Windows in between.

The best of both worlds is to use your Mac to access cloud computing, I'm not quite sure why you're having trouble installing your copy of MATLAB wherever you like, but maybe that's because I'm not familiar enough with the licensing details.

[Kevin Johnson]

Joss,

Ha. I've done that first method a few times myself...

My 'home use" license doesn't permit cloud computing. Going to an individual commercial license would cost somewhere between $10,000 to $20,000 once all the toolboxes etc.are accounted for. This is a hobby, so I can't afford that much. More precisely, I can't figure out how to convince my wife I can afford that much. Oh well. Thank god somebody has sense.

Best,

Kevin

[Walter Roberson]

I just ran bench on a Macbook Pro with M1 Max. The speed was 40... same as my older Intel iMac.