lazy from typing import Iterator
def stream_events(...) -> Iterator[str]:
while True:
yield blocking_get_event(...)
events = stream_events(...)
for event in events:
consume(event)
Do we finally have "lazy imports" in Python? I think I missed this change. Is this also something from Python 3.15 or earlier?Check out symmetric difference
I can see one way forward being to prototype them in python and convert.
> This is an unfortunate problem I've encountered many times, and it's often a problem for normal decorators too. But this has changed in 3.15, now the ContextDecorator will check the type of the function it's wrapping and ensure that the decorator covers the entire lifespan.
I very much like the idea of that change - but it also seems kind of dangerous, to do this with no "opt-in mechanism", as that quite subtly changes the behavior of existing usage sites.
This is a bit of a "spacebar heating" situation, because someone would have to intentionally use a decorator in the old, broken way, but if someone actually did that, things may unexpectedly break.
Try and write a signal processing thing with filters, windowing, overlap, etc. - there's no easy way to do it at all with the libraries that exist.
lazy from typing import Iterator
def stream_events(...) -> Iterator[str]: while True: yield blocking_get_event(...)
events = stream_events(...)
for event in events: consume(event)
Lazy loading looks like a last nail in the coffin, where my love to Python was buried, although it was a long, tiresome process.
>>> from collections import Counter
>>> c = Counter(a=3, b=1)
>>> d = Counter(a=1, b=2)
>>> c-d
Counter({'a': 2})But damn, with all the supply chain attacks now in the news, could they just make a simple way (for non python insiders) to install python apps without fearing to be infected by a vermin with full access to my $HOME ...
And now even type imports are apparently so slow that you have to disable them if unused during the normal untyped execution.
If Instagram or others wants a professional language, they should switch to Go or PHP instead of shoehorning strange features into a language that wasn't built for their use cases.
If they only considered parities it could be interpreted as addition in F_2, which is more natural, but I'd still agree that it's hard to see how you'd use something like this in practice.
Once your program starts to get bigger you have abstractions that can cope fairly well and keep your code simple to use - this is what Perl didn't have.
If you need more speed then you can write extensions in some compiled language.I think TCL was better at this hybrid approach but Python is a nicer language in itself.
You can also just dump python and write everything in that other language but now you understand the problem space quite well and you won't be trying to learn about it using a language where change is "difficult."
https://peps.python.org/pep-0649/
https://docs.python.org/3/reference/compound_stmts.html#anno...
With 3.15, using lazy typing imports is more or less an alternative to putting such imports behind an "if TYPE_CHECKING" guard.
There are ways to harden and/or reduce privileges, but shells/scripting languages will always have this issue on any modern OS.
The UNIX way to help prevent that is really to run processes as another user, but people seem to refuse to do so. You should always expect any process running as your UID to be able to access any data owned or visible to your UID.
While it is possible to reduce the risk of disclosure, they are all wack-a-mole preventions protecting the low hanging fruit, not absolute guarantees.
That is purely due to how UNIX works [0]
[0] https://man7.org/linux/man-pages/man7/credentials.7.html
All of our services we were our are significantly faster and more reliable. We used Rust, it wasn’t hard to do
- You wrote 100K lines of code (I've worked on several large C++ projects that were far smaller)
- You wrote those lines in Python (surely the whole point of Python is to write less code)
- You deleted them (never delete anything, isn't this what modern VCS is all about?)
But whatever floats your boat.
Just because you don’t like a feature doesn’t mean it’s because of AI and bad code.
The lazy statement would be an improvement as it allows one to see all the imports at the top where you expect them to be.
1. The very first steps are quite simple. Hello world is literally just `print("hello world")`. In other languages it can be a lot more complex.
2. It got a reputation as a beginner-friendly language as a result.
3. It has a "REPL" which means you can type code into a prompt and it will execute it interactively. This is very helpful for research (think AI) where you're trying stuff out and want to plot graphs and so on.
IMO it is undeservedly popular, or at least was. Wind back 10 years to when it was rapidly gaining mindshare:
1. While "hello world" is simple, if you went further to more complex programs you would hit two roadblocks: a) the lack of static type checking means large programs are difficult to maintain, and b) it's really really slow.
2. While the language is reasonable, the tooling (how you install packages, manage the code and so on) was eye-bleedingly abysmal.
3. While the REPL did technically exist, it was really bare bones. It couldn't even handle things like pasting code into it if the code contained blank lines (which it usually does).
However since it has become arguably the most popular language in the world, a lot of people have been forced to use it and so it is actually getting quite decent now. It has decent static types (even if lots of people still don't use them), the REPL is actually decent now (this changed very recently), and there's a new third party tool called `uv` to manage your code that is actually good.
The biggest issue with it now is that it's still horrifically slow (around 50-200x slower than "fast" languages like C++, Rust etc). It is pretty unlikely that that will ever change. People always try to excuse this by saying Python is a "glue" language and you just use it to connect components written in faster languages, but a) that's pure "you're holding it wrong", and b) that only works in some cases where there are nicely separated "slow bits" that can be moved to another language. That's the case for AI for example, where it's all numerical, but for lots of things it isn't. Mercurial was a competitor to Git that was written in Python and lost partly because it was way too slow. They've started writing parts in Rust but it took them 10 years to even start doing that and by then it was far too late.
> what would you suggest?
It really depends on what you want to make. I would pick something to make first and then pick the language based on that. Something like:
* AI: Python for sure. Make sure you use uv and Pyright.
* Web-based games: Typescript
* Web sites: Typescript, or maybe Go.
* Desktop GUI: Tbh I'd still use C++ with QtWidgets. Getting a bit old-school now tbf.
Also Rust is the best language of them all, but I dunno if I'd pick it as a beginner unless you really know you want to get into programming.
Oh, that is such a nice thing.
I personally chose C# for this reason, because ASP.NET is mature and (IMO) well designed. But there's also Java/Spring and and lots of other options in different languages depending on your preferences.
The very idea that you offer a (python) package installer that is gonna pull a tree of code published and updated by random people in an unvetted manner open the door to all the supply chain attacks we are seeing.
Around the same time (early 90s) Java was designed with high isolation in mind but the goal and vision was very different. And Java had its own problems.
I'm saying that because at some point the security problem is gonna really hurt the python ecosystem.
Yes it strains at the big to huge project end, not recommended to take it there. Still there are better tools to help now.
lol, no. Just no. Python is far superior for website backends unless perhaps you're running one of the top 20 websites in the world.
Python-dev now is paid to shore up the failed Instagram stack.
The person said: "deleted 100k+ lines this year already moving them to faster languages"
Are you saying that when you move code to another language/rewrite in another language, you leave the original languages code in your repo?
They didn't say they deleted it from their git history. I delete code all the time (doesn't mean its "gone", just that its not in my git head).
Our entire business runs on 300k lines of Ruby (on Rails) and I can keep most of the business logic in my head. I would say our codebase is not exactly “tiny” and just cracking the ceiling into “smal” territory. And comparatively, people probably write even less code in equivalent rails apps to django ones. 100k lines of C++ is miniscule.
Obviously “deleting code” in this context doesn’t mean purging version control history but the current state of the codebase.
Choosing a language is a game of trade-offs: potentially slower execution in return for faster development time, for example. If your team is already familiar with Ruby, will asking them to write a project in Rust necessarily result in a better product? Maybe, but it will almost certainly take much longer.
Anyway, how many Python programs are actually "too slow"? Most of the time, Python is fast enough, even if heavy computation is offloaded to other languages.
As for Rust being the best language of them all, that's, like, your opinion, man.
The example:
>> 'hello'.toUpperCase()
Traceback (most recent call last):
...
AttributeError: 'str' object has no attribute 'toUpperCase'. Did you mean '.upper'?> from __future__ import annotations (PEP 563) will continue to exist with its current behavior at least until Python 3.13 reaches its end-of-life. Subsequently, it will be deprecated and eventually removed.
Indentation is a horrible decision (there’s a reason no other language went this way), which led to simple concepts like blocks/lambdas having pretty wild constraints (only one line??)
Type decoration has been a welcome addition, but too slowly iterated on and the native implementations (mypy) are horribly slow at any meaningful size.
Concurrency was never good and its GIL+FFI story has boxed it into a long-term pit of sadness.
I’ve used it for years, but I’m happy to see it go. It didn’t win because it was the best language.
If you mean "easy to get something out of it" then yeah, it's great.
I personally now use a mixture of Typescript and Rust for most things, including AI coding. Its been working quite well. (AI doesn't handle Rust as well as TS, in that the code isn't quite idiomatic, but it does ok)
in the case of lazy imports the big corps were the ones doing the experimentation and iteration. the feature didn't make it into the language "just when big corps wanted them"; the instagram stack you allude to already had its own fork of cpython with lazy imports added years ago, and that is not the design that ended up getting adopted by upstream cpython, though some of the people working on it also collaborated on the PEP that finally did make it in.
I would need some evidence of that.
That being said, I'm starting all new large development work in Rust. Python is hard to reason about due to its dynamic nature in large codebases. And if I'm enabling strict typing everywhere, I might as well use a typed language and get a performance boost. Obviously, this is only because I'm the sole developer and using AI to improve productivity.
Work settings are completely different and one has to be a team player to find the language that works for everyone.
Another classic lie about Python. The slower speed doesn't matter because it's development speed that's important, and Python gives you faster development speed!
Except... it absolutely doesn't. It would be very difficult to argue that Typescript has significantly slower development speed but it is much faster to execute. I also disagree that Python is any faster than Go, Rust or Lotion, but I think lots of people blindly accept that it is and would argue based on that.
No, no, it is not, or at least not in my experience (I do not and never have done web development - medium performance C++ code - I don't see how I could write, understand and support 100K lines of code in this area).
And so, what does your Ruby code actually do?
I really appreciate them going out of their way to do this, being quite aware of the hidden complexity in doing it.
I will never understand why people are upset about this.
You HAVE multi-line lambdas. They're called functions.
Yeah, I know you want a function that's only used once to be able to be defined in-line, but tbh I've always found that syntax to be pretty ugly, especially once you're passing two functions to a single call, or have additional parameters AFTER the function (I'm looking at you, setTimeout/setInterval).
Counter-example is Go and init() function.
Except of course for those that did, Haskell, Fortran for example.
Deleting "from my codebase" doesn't imply deleting it from history or backups. Just that the code isn't present for future edits or deployments.
The way you're talking, it sounds like you never delete code from your codebase. Do you just comment it out when you change a line to something else or replace a function with a new one? Just add new files?
The versioning issue I've seen across libraries that version change in many languages.
I don't tend to hit Python 2 issues using LLMs with it, but I do hit library things (e.g. Pydantic likes to make changes between libraries - or loads of the libraries used a lot by AI companies).
Ironically, the new default behavior (making type annotation evaluation lazy) is not backwards compatible with the "from __future__ import annotations" behavior of converting annotations to strings, so they can't just rip out "from __future__ import annotations" and instead it needs to be deprecated and removed over multiple releases.
Oh, what tangled webs we weave! :-)
But this is a "...will continue to exist with its current behavior at least..." is an important bit there.
From pep-0749:
Sometime after the last release that did not support PEP 649 semantics (expected to be 3.13) reaches its end-of-life, from __future__ import annotations is deprecated. Compiling any code that uses the future import will emit a DeprecationWarning. This will happen no sooner than the first release after Python 3.13 reaches its end-of-life, but the community may decide to wait longer.
It has a good overview of the history.Because the alternative is a rather sociopathic level of schadenfreude.
It’s pretty ok in Python, but meaningful indentation is amazing with a proper type system and compiler. Clean, consistent, efficient, and ensures working code is easily read and standardized.
I’m unaware of anyone accepting improperly formatted C# as ‘done’, and would reject any such PR out of hand because of the potential for legibility issues to hide bugs. So: if it were done when 'tis done, then 'twere well it were done by the compiler to save line noise.
I also don’t have issues with quality of Python generated. It takes a bit of nudging to use list comps and generators rather than imperative forms but it tends to mimic code already in context. So if the codebase is ok, it does do better.
Most languages have this feature Afaik
If you’re asking whether project leads give more weight to a single, tangible, vocal stakeholder than they do to unknown numbers of anonymous and lightly-engaged stakeholders? Yes.
Our experiences differ then. Mine is that almost all of the code I write is directly targeted on the usually quite complex problem I am trying to solve. I don't do boilerplate, for example.
(To preempt potential pedantry: yes, I know that you can compile Python to bytecode ahead of time, but that's not really relevant to what's being discussed here because it doesn't mean "the stuff happening in modules I import isn't happening at runtime anymore")
"I don't think it makes much sense to try to add "functional" primitives to Python, because the reason those primitives work well in functional languages don't apply to Python, and they make the code pretty unreadable for people who aren't used to functional languages (which means most programmers). I also don't think that the current crop of functional languages is ready for mainstream." https://developers.slashdot.org/story/13/08/25/2115204/inter...
Example: to build our system, you run `mach build`. For faster rebuilds, you can do `mach build <subdir>`, but it's unreliable. AI agents love to use it, often get errors that would be fixed by a full-tree build, and will chase their tails endlessly trying to fix things that aren't broken. So someone turned off that capability by default and added a flag `--allow-subdirectory-build` for if you want to use it anyway. So that people would know about it, they added a helpful warning message pointing you to the option[1].
The inevitable (in retrospect) happened: now the AI would try to do a subdirectory build, it would fail, the AI would see the warning message, so it would rerun with the magic flag set.
So now the warning message is suppressed when running under an AI[2][3]. The comment says it all:
# Don't tell agents how to override, because they do override
"The user does not want me to create the Torment Nexus but did not specify why it would be a problem, so I will first create the Torment Nexus in order to understand the danger of creating the Torment Nexus."[1] https://searchfox.org/firefox-main/rev/fc94d7bda17ecb8ac2fa9...
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=2034163
[3] https://searchfox.org/firefox-main/rev/cebc55aab4d2661d1f6c2...
It's that time of the year again, a new version of Python is just around the corner. With the Python 3.15.0b1 feature freeze, we know what's coming to Python later this year. There are so many big features coming including lazy imports and the tachyon profiler which I previously covered.
Last year, I really enjoyed investigating the smaller features of Python 3.14. I found that many of those features were just as interesting as the big PEPs and deserve a lot more attention. This year the situation is no different.
There are not many Asyncio changes in this releases. The main feature to come out here is the ability to cancel a TaskGroup gracefully.
TaskGroup is a form of structured concurrency, it enables developers to create multiple concurrent tasks in a clean way.
async with asyncio.TaskGroup() as tg: tg.create_task(run()) tg.create_task(run()) # Waits for all the tasks to complete
Suppose we want to wait in the background for a signal of sorts to interrupt the taskgroup's execution, it's seems like something simple to do in asyncio, but in reality it's somewhat awkward to do this.
class Interrupt(Exception): ... with suppress(Interrupt): async with asyncio.TaskGroup() as tg: tg.create_task(run()) tg.create_task(run()) if await wait_for_signal(): raise Interrupt()
This works because exceptions raised within a task group cause other tasks to cancel. The custom Interrupt exception is raised as part of a ExceptionGroup which then gets filtered by contextlib.suppress, resulting in a graceful exit.
The way suppress works with ExceptionGroup is yet another overlooked feature from 3.12. This is a change I learnt by accident when researching this article.
The new TaskGroup.cancel makes this process a lot easier:
async with asyncio.TaskGroup() as tg: tg.create_task(run()) tg.create_task(run()) if await wait_for_signal(): tg.cancel()
Unlike before it's so simple there's hardly any point in explaining. It simply cancels the group without raising any exceptions.
Decorators are surprisingly hard to write, so much so that it's become a go-to interview question. But did you know that context managers can also double up as a decorator?
@contextmanager def duration(message: str) -> Iterator[None]: start = time.perf_counter() try: yield finally: print(f"{message} elapsed {time.perf_counter() - start:.2f} seconds")
Here I have a very commonly used context manager to print out the duration spent in the block. Ever since Python 3.3 we could directly use it as a decorator too:
@duration('workload') def workload(): ... # Or simple as a wrapper duration('stuff')(other_workload)(...)
But whilst it's convenient, there are cases where it doesn't work at all:
@duration('async workload') async def async_workload(): ... @duration('generator workload') def workload(): while True: yield ...
Iterators, async functions and async iterators don't work well here because they have different semantics to standard functions. When you call them they return immediately with a generator object, coroutine function and async generator object respectively. So the decorator completes immediately as opposed to the entire lifecycle what it's wrapping.
This is an unfortunate problem I've encountered many times, and it's often a problem for normal decorators too. But this has changed in 3.15, now the ContextDecorator will check the type of the function it's wrapping and ensure that the decorator covers the entire lifespan.
In my opinion, this now makes context managers the best way to create decorators! It avoids some of the common footguns and provides cleaner syntax. I recommend more people start using it this way.
Iterators are one of the foundations of modern Python. The iterator type allows us to separate data sources from data consumers as below, resulting in cleaner abstractions:
lazy from typing import Iterator def stream_events(...) -> Iterator[str]: while True: yield blocking_get_event(...) events = stream_events(...) for event in events: consume(event)
But this abstraction breaks when using threading or free-threading. An iterator by default is not threadsafe, therefore we may see skipped values or just broken internal iterator state.
This is solved in 3.15 with threading.serialize_iterator, we simply wrap our original iterator with this and voila:
import threading events = threading.serialize_iterator(stream_events(...)) with ThreadPoolExecutor() as executor: fut1 = executor.submit(consume, events) fut2 = executor.submit(consume, events)
There is also the threading.synchronized_iterator decorator which just applies threading.serialize_iterator to the result of an generator function.
Finally we also have threading.concurrent_tee that instead of splitting the values will duplicate the values across multiple iterators:
source1, source2 = threading.concurrent_tee(squares(10), n=2) with ThreadPoolExecutor() as executor: fut1 = executor.submit(consume, source1) fut2 = executor.submit(consume, source2)
Before these utilities existed we primarily relied on Queues to synchronise consumption between threads, with these added in we can avoid changing our abstractions for multi-threaded code.
Last year I only highlighted 3 features, but this year there are a lot more updates that intrigue me. Here are 2 more changes that are perhaps less impactful but still very interesting nonetheless.
collections.Counter is a very useful class. It let's us easily count up the frequency of discrete occurrences. It behaves very similar to a dict[KeyType, int] but with a ton of useful operations
c = Counter(a=3, b=1) d = Counter(a=1, b=2) print(f"{c + d = }") # add two counters together: c[x] + d[x] print(f"{c - d = }") # subtract (keeping only positive counts)
prints:
Counter(a=4, b=3) Counter(a=1, b=0)
But it has some weirder operations too:
print(f"{c & d = }") # intersection: min(c[x], d[x]) print(f"{c | d = }") # union: max(c[x], d[x])
prints:
Counter(a=1, b=1) Counter(a=3, b=2)
The way to think of it is that a Counter can also represents a discrete set of objects. so in our example, we're essentially doing:
{a_0, a_1, a_2, b_0} & {a_0, b_0, b_1} == {a_0, b_0} {a_0, a_1, a_2, b_0} | {a_0, b_0, b_1} == {a_0, a_1, a_2, b_0, b_1}
In 3.15 we can also add xor to the list:
c = Counter(a=3, b=1) d = Counter(a=1, b=2) c ^ d == c | d - c & d == Counter(a=3, b=2) - Counter(a=1, b=1) == Counter(a=2, b=1)
Once again this is best explained by our notation from earlier:
{a_0, a_1, a_2, b_0} ^ {a_0, b_0, b_1} == {a_1, a_2, b_1}
I've left this one to the bonus section because I've never used set operations on Counters and I'm finding it extremely hard to think of a use case for xor specifically. But I do appreciate the devs adding it for completeness.
With the addition of frozendict in 3.15, we now have the ability to represent all the json types (array, boolean, float, null, string, object) in immutable (hashable) forms.
A change has been made to json.load and json.loads to add array_hook parameter that compliments the object_hook parameter. This now allows us to parse json objects directly into this form:
json.loads('{"a": [1, 2, 3, 4]}', array_hook=tuple, object_hook=frozendict) == frozendict({'a': (1, 2, 3, 4)})
Proudly powered by Pelican, which takes great advantage of Python.
The theme is by Smashing Magazine, thanks!