I wonder how much memory can Python hold until an error like “out of memory” happens, because ML models (for example, those hosted and served in HuggingFace) loads training weights with dozens of GBs

All the stuff that’s LLM and the actual “serious” python libraries are implemented in C/C++ and only made accessible via python.

Which doesn’t directly answer the question of what the maximum is, in those cases, but it should be obvious that C/C++ have some good ways to deal with memory.

You can still do “traditional” memory management in python, or “memory aware programming” like, e.g. not trying to read a file in one piece, but reading and processing line by line.

And using C from python is actually very easy and convenient with ctypes. https://docs.python.org/3/library/ctypes.html

Why the heck would 2 projects share the same library?

Coming from the olden days, with good package management, infrequent updates and the idea that you wanted to indeed save that x number of bytes on the disk and in memory, only installing one was the way to go.

Python also wasn’t exactly a high brow academic effort to brain storm the next big thing, it was built to be a simple tool and that included just fetching some library from your system was good enough. It only ended up being popular because it is very easy to get your feet wet and do something quick.

The difficulty with python tooling is that you have to learn which tools you can and should completely ignore.

Unless you are a 100x engineer managing 500 projects with conflicting versions, build systems, docker, websites, and AAAH…

• you don’t really need venvs
• you should not use more than on package manager (I recommend pip) and you should cling to it with all your might and never switch. Mixing e.g. conda, on linux system installers like apt, is the problem. Just using one is fine.
• You don’t “need” need any other tools. They are bonuses that you should use and learn how to use, exactly when you need them and not before. (type hinting checker, linting, testing, etc…)

Why is it like this?

Isolation for reliability, because it costs the businesses real $$$ when stuff goes down.

venvs exists to prevent the case that “project 1” and “project 2” use the same library “foobar”. Except, “project 1” is old, the maintainer is held up and can’t update as fast and “project 2” is a cutting edge start up that always uses the newest tech.

When python imports a library it would use “the libary” that is installed. If project 2 uses foobar version 15.9 which changed functionality, and project 1 uses foobar uses version 1.0, you get a bug, always, in either project 1 or project 2. Venvs solve this by providing project specific sets of libraries and interpreters.

In practice for many if not most users, this is meaningless, because if you’re making e.g. a plot with matplotlib, that won’t change. But people have “best practices” so they just do stuff even if they don’t need it.

It is a tradeoff between being fine with breakage and fixing it when it occurs and not being fine with breakage. The two approaches won’t mix.

very specific (often outdated) version of python,

They are giving you the version that they know worked. Often you can just remove the specific version pinning and it will work fine, because again, it doesn’t actually change that much. But still, the project that’s online was the working state.

I have a meme answer to this.

https://www.youtube.com/watch?v=Gn51xYvJhQI

It’s not that they are unfriendly.

But they are 100% there to represent the company’s interest and not yours. If there is any way, to… turn a situation into something where the company gets more money out of it and you get less, it’s their job to make that happen.

In theory they should have employee retention in mind. In practice, nobody does their HR that way anymore.

All my interactions with HR have been “professional polite” and appropriately friendly. There is no reason to be unnecessarily mean, they are also just doing their job.

Some kind of general fitness testing?

You know, involving heart, lung capacity, performance?

All the ones where the idea was to “just start something, grow grow grow, then figure out monetization later” is wild to me.

E.g. reddit. It worked. CEO is rich, site is still online. Somehow they got investors probably, presumably.

I get not having profit. I get not having income, if it’s in some prototype phase. But having no plan or idea whatsoever for how to monetize and still getting VC? Wild.

I had a phase as a teen when I was constantly swearing. My parents told me that, it can’t be that bad and it’s really annoying.

And it’s mostly an impulse reaction and we’re kind of above that.

It doesn’t mean that you can’t express pain or anger. You’re just not insulting people’s ears if you scream “Aaaaah” when you bang your toe against a table leg or something. And your environment really doesn’t deserve it. Most people are somewhat compassionate and you’re just swearing while they try to help… that’s not a pleasant environment for them to be in. It makes it harder to help you.

No to both questions. I just made a change and that was it. And it has never stopped me from expressing anything.

If anything, it lends more weight to the regular words.

A _______ criminal? Or a criminal?

You can still put the same emotion into the words, they’re just not swear words. :)

At the cost of sounding naive and stupid

It may be a naive question, but it’s a very important naive question. Naive doesn’t mean bad.

The answer is that that is not possible, because the compiler is supposed to translate the very specific language of C into mostly very specific machine instructions. The programmers who wrote the code, did so because they usually expect a very specific behavior. So, that would be broken.

But also, the “unsafety” is in the behavior of the system and built into the language and the compiler.

It’s a bit of a flawed comparison, but you can’t build a house on a foundation of wooden poles, because of the advantages that wood offers, and then complain that they are flammable. You can build it in steel, but you have to replace all of the poles. Just the poles on the left side won’t do.

And you can’t automatically detect the unsafe parts and just patch those either. If we could, we could just fix them directly or we could automatically transpile them. Darpa is trying that at the moment.

I always recommend this one:

https://www.w3schools.com/python/default.asp

Because the topics are sorted, easy to find and bite sized.