Planet Python

Last update: April 07, 2025 01:43 PM UTC

April 05, 2025

The word2vec model was proposed in a 2013 paper by Google researchers called "Efficient Estimation of Word Representations in Vector Space", and was further refined by additional papers from the same team. It kick-started the modern use of embeddings - dense vector representation of words (and later tokens) for language models.

Also, the code - with some instructions - was made available openly. This post reproduces the word2vec results using JAX, and also talks about reproducing it using the original C code (see the Original word2vec code section for that).

Embeddings

First, a brief introduction to embeddings. Wikipedia has a good definition:

In natural language processing, a word embedding is a representation of a word. The embedding is used in text analysis. Typically, the representation is a real-valued vector that encodes the meaning of the word in such a way that the words that are closer in the vector space are expected to be similar in meaning

Here's a framework that made sense to me when I was first learning about embeddings many years ago:

ML models and NNs specifically are all about vector math.
Words in a human language (like English) are just sequences of characters with no semantic meaning (there's nothing in the word "dog" that conveys dog-ness any more than the same concept in other human languages). Also, words have different lengths which isn't convenient.
To represent words as vectors, we typically use indices into a vocabulary; equivalently, this can be seen as a one-hot vector with the value at the correct vocabulary index being 1, and the rest 0.
This latter vector representation has no semantic meaning either, because "Paris" and "France" will be as different from each other as "Paris" and "Armadillo". Also, these vectors are huge (a typical vocabulary can have tens of thousands of words, just for a single language!)
Therefore, we need some magic to convert words into vectors that carry meaning.

Embeddings are that magic. They are dense vectors of floats - with typically hundreds or thousands of elements, and serve as representations of these words in high-dimensional space.

The word2vec CBOW architecture

The word2vec paper proposed two related architectures: CBOW (Continuous Bag Of Words) and Continuous Skip Gram. The two are fairly similar, and in this post I'm going to focus on CBOW.

The idea of the CBOW approach is to teach the model to predict a word from its surrounding words. Here's an example with window size of four [1]:

CBOW - showing word in center of window, with context words around

The goal here is to have the model predict that "liberty" should be the word in the middle, given the context words in peach-colored boxes. This is an unsupervised model - it learns by consuming text, sliding its window word by word over arbitrary amounts of (properly formatted and sanitized) input.

Concretely, the following diagram shows the model architecture; here are the dimensions involved:

B: batch (for computational efficiency, whole batches are processed together)
V: vocabulary size (the number of unique words in our vocabulary)
D: model depth (the size of the dense embedding vectors we're trying to learn)
W: window size

Here's the flow of data in the forward pass:

context is the context words for a given position. For example, in the sample diagram above the context would be of length 8. Each element is an integer representation of a word (its index into the vocabulary). Since we're processing batches, the shape of this array is (B,2W).
The context indexes into a projection matrix P, which has the learned embedding per row - one for each word in the vocabulary. The result is projection with shape (B,2W,D). The first two dimensions remain the same (because we still have the same batch and window size), but every integer is replaced with the word's embedding - so an extra dimension is added.
Next, a mean (arithmetic average) is taken across the window dimension. The embeddings of all the words in the window are averaged together. The result is (B,D) where each row is the average of the embeddings of 2W words.
Finally, the hidden layer matrix H is used to map the dense representation back into a sparse one [2] - this is the prediction of the middle word. Recall that this tries to predict a one-hot encoding of the word's vocabulary index.

For training, the loss is calculated by comparing out to the one-hot encoding of the actual target word for this window, and the calculated gradient is propagated backwards to train the model.

JAX implementation

The JAX implementation of the model described above is clean and compact:

@jax.jit
def word2vec_forward(params, context):
    """Forward pass of the word2Vec model.

    context is a (batch_size, 2*window_size) array of word IDs.

    V is the vocabulary size, D is the embedding dimension.
    params["projection"] is a (V, D) matrix of word embeddings.
    params["hidden"] is a (D, V) matrix of weights for the hidden layer.
    """
    # Indexing into (V, D) matrix with a batch of IDs. The output shape
    # is (batch_size, 2*window_size, D).
    projection = params["projection"][context]

    # Compute average across the context word. The output shape is
    # (batch_size, D).
    avg_projection = jnp.mean(projection, axis=1)

    # (batch_size, D) @ (D, V) -> (batch_size, V)
    hidden = jnp.dot(avg_projection, params["hidden"])
    return hidden


@jax.jit
def word2vec_loss(params, target, context):
    """Compute the loss of the word2Vec model."""
    logits = word2vec_forward(params, context)  # (batch_size, V)

    target_onehot = jax.nn.one_hot(target, logits.shape[1])  # (batch_size, V)
    loss = optax.losses.softmax_cross_entropy(logits, target_onehot).mean()
    return loss

Training

For training, I've been relying on the same dataset used by the original word2vec code - a 100MB text file downloaded from http://mattmahoney.net/dc/text8.zip

This file contains all-lowercase text with no punctuation, so it requires very little cleaning and processing. What it does require for higher-quality training is subsampling: throwing away some of the most common words (e.g. "and", "is", "not" in English), since they appear so much in the text. Here's my code for this:

def subsample(words, threshold=1e-4):
    """Subsample frequent words, return a new list of words.

    Follows the subsampling procedure described in the paper "Distributed
    Representations of Words and Phrases and their Compositionality" by
    Mikolov et al. (2013).
    """
    word_counts = Counter(words)
    total_count = len(words)
    freqs = {word: count / total_count for word, count in word_counts.items()}

    # Common words (freq(word) > threshold) are kept with a computed
    # probability, while rare words are always kept.
    p_keep = {
        word: math.sqrt(threshold / freqs[word]) if freqs[word] > threshold else 1
        for word in word_counts
    }
    return [word for word in words if random.random() < p_keep[word]]

We also have to create a vocabulary with some limited size:

def make_vocabulary(words, top_k=20000):
    """Creates a vocabulary from a list of words.

    Keeps the top_k most common words and assigns an index to each word. The
    index 0 is reserved for the "<unk>" token.
    """
    word_counts = Counter(words)
    vocab = {"<unk>": 0}
    for word, _ in word_counts.most_common(top_k - 1):
        vocab[word] = len(vocab)
    return vocab

The preprocessing step generates the list of subsampled words and the vocabulary, and stores them in a pickle file for future reference. The training loop uses these data to train a model from a random initialization. Pay special attention to the hyper-parameters defined at the top of the train function. I set these to be as close as possible to the original word2vec code:

def train(train_data, vocab):
    V = len(vocab)
    D = 200
    LEARNING_RATE = 1e-3
    WINDOW_SIZE = 8
    BATCH_SIZE = 1024
    EPOCHS = 25

    initializer = jax.nn.initializers.glorot_uniform()
    params = {
        "projection": initializer(jax.random.PRNGKey(501337), (V, D)),
        "hidden": initializer(jax.random.PRNGKey(501337), (D, V)),
    }

    optimizer = optax.adam(LEARNING_RATE)
    opt_state = optimizer.init(params)

    print("Approximate number of batches:", len(train_data) // BATCH_SIZE)

    for epoch in range(EPOCHS):
        print(f"=== Epoch {epoch + 1}")
        epoch_loss = []
        for n, (target_batch, context_batch) in enumerate(
            generate_train_vectors(
                train_data, vocab, window_size=WINDOW_SIZE, batch_size=BATCH_SIZE
            )
        ):
            # Shuffle the batch.
            indices = np.random.permutation(len(target_batch))
            target_batch = target_batch[indices]
            context_batch = context_batch[indices]

            # Compute the loss and gradients; optimize.
            loss, grads = jax.value_and_grad(word2vec_loss)(
                params, target_batch, context_batch
            )
            updates, opt_state = optimizer.update(grads, opt_state)
            params = optax.apply_updates(params, updates)

            epoch_loss.append(loss)
            if n > 0 and n % 1000 == 0:
                print(f"Batch {n}")

        print(f"Epoch loss: {np.mean(epoch_loss):.2f}")
        checkpoint_filename = f"checkpoint-{epoch:03}.pickle"
        print("Saving checkpoint to", checkpoint_filename)
        with open(checkpoint_filename, "wb") as file:
            pickle.dump(params, file)

The only thing I'm not showing here is the generate_train_vectors function, as it's not particularly interesting; you can find it in the full code.

I don't have a particularly powerful GPU, so on my machine training this model for 25 epochs takes 20-30 minutes.

Extracting embeddings and finding word similarities

The result of the training is the P and H arrays with trained weights; P is exactly the embedding matrix we need! It maps vocabulary words to their dense embedding representation. Using P, we can create the fun word demos that made word2vec famous. The full code has a script named similar-words.py that does this. Some examples:

$ uv run similar-words.py -word paris \
      -checkpoint checkpoint.pickle \
      -traindata train-data.pickle
Words similar to 'paris':
paris           1.00
france          0.50
french          0.49
la              0.42
le              0.41
henri           0.40
toulouse        0.38
brussels        0.38
petit           0.38
les             0.38

And:

$ uv run similar-words.py -analogy berlin,germany,tokyo \
      -checkpoint checkpoint.pickle \
      -traindata train-data.pickle
Analogies for 'berlin is to germany as tokyo is to ?':
tokyo           0.70
japan           0.45
japanese        0.44
osaka           0.40
china           0.36
germany         0.35
singapore       0.32
han             0.31
gu              0.31
kyushu          0.31

This brings us to the intuition for how word2vec works: the basic idea is that semantically similar words will appear in the vicinity of roughly similar context words, but also that words are generally related to words in the context their appear in. This lets the model learn that some words are more related than others; for example:

$ uv run similar-words.py -sims soccer,basketball,chess,cat,bomb \
      -checkpoint checkpoint.pickle \
      -traindata train-data.pickle
Similarities for 'soccer' with context words ['basketball', 'chess', 'cat', 'bomb']:
basketball      0.40
chess           0.22
cat             0.14
bomb            0.13

Optimizations

The word2vec model can be optimized in several ways, many of which are focused on avoiding the giant matrix multiplication by H at the very end. The word2vec authors have a followup paper called "Distributed Representations of Words and Phrases and their Compositionality" where these are described; I'm leaving them out of my implementation, for simplicity.

Implementing these optimizations could help us improve the model's quality considerably, by increasing the model depth (it's currently 200, which is very low by modern LLM standards) and the amount of data we train on. That said, these days word2vec is mostly of historical interest anyway; the Modern text embeddings section will have more to say on how embeddings are trained as part of modern LLMs.

Original word2vec code

As mentioned above, the original website for the word2vec model is available on an archived version of Google Code. That page is still useful reading, but the Subversion instructions to obtain the actual code no longer work.

I was able to find a GitHub mirror with a code export here: https://github.com/tmikolov/word2vec (the username certainly checks out, though it's hard to know for sure!)

The awesome thing is that this code still builds and runs perfectly, many years later. Hurray to self-contained C programs with no dependencies; all I needed was to run make, and then use the included shell scripts to download the data and run training. This code uses the CPU for training; it takes a while, but I was able to reproduce the similarity / analogy results fairly easily.

Modern text embeddings

The word2vec model trains an embedding matrix; this pre-trained matrix can then be used as part of other ML models. This approach was used for a while, but it's no longer popular.

These days, an embedding matrix is trained as part of a larger model. For example, GPT-type transformer-based LLMs have an embedding matrix as the first layer in the model. This is basically just the P matrix from the diagram above [3]. LLMs learn both the embeddings and their specific task (generating tokens from a given context) at the same time. This makes some sense because:

LLMs process enormous amounts of data, and consuming this data multiple times to train embeddings separately is wasteful.
Embeddings trained together with the LLM are inherently tuned to the LLM's specific task and hyper-parameters (i.e. the kind of tokenizer used, the model depth etc.)

Specifically, modern embedding matrices differ from word2vec in two important aspects:

Instead of being word embeddings, they are token embeddings. I wrote much more on tokens for LLMs here.
The model depth (D) is much larger; GPT-3 has D=12288, and in newer models it's probably even larger. Deep embedding vectors help the models capture more nuance and semantic meaning about tokens. Naturally, they also require much more data to be trained effectively.

Full code

The full code for this post is available here. If you want to reproduce the my word2vec results, check out the README file - it contains full instructions on which scripts to run and in which order.

[1]	The window size is how many words to the left and right of the target word to take into account, and it's a configurable hyper-parameter during training.

[2]

The terms dense and sparse are used in the post in the following sense:

A sparse array is one where almost all entries are 0. This is true for one-hot vectors representing vocabulary words (all entries are 0 except a single one that has the value 1).

A dense array is filled with arbitrary floating-point values. An embedding vector is dense in this sense - it's typically short compared to the sparse vector (in the word2vec example used in this post D=200, while V=20000), but full of data (hence "dense"). An embedding matrix is dense since it consists of dense vectors (one per word index).

[3]	The rest (mean calculation, hidden layer) isn't needed since it's only there to train the word2vec CBOW model.

April 05, 2025 08:18 PM UTC

Python Engineering at Microsoft

Build AI agents with Python in #AgentsHack

2025 is the year of AI agents! But what exactly is an agent, and how can you build one? Whether you’re a seasoned developer or just starting out, this free three-week virtual hackathon is your chance to dive deep into AI agent development.

Throughout the month of April, join us for a series of live-streamed sessions on the Microsoft Reactor YouTube channel covering the latest in AI agent development. Over twenty streams will be focused on building AI agents with Python, using popular frameworks like Semantic Kernel, Autogen, and Langchain, as well as the new Azure AI Agent Service.

Once you’ve learned the basics, you can put your skills to the test by building your own AI agent and submitting it for a chance to win amazing prizes.

The hackathon welcomes all developers, allowing you to participate individually or collaborate in teams of up to four members. You can also use any programming language or framework you like, but since you’re reading this blog, we hope you’ll consider using Python!

Register now! Afterwards, browse through the live stream schedule below and register for the sessions you’re interested in.

Live streams

You can see more streams on the hackathon landing page, but below are the ones that are focused on Python. You can also sign up specifically for the Python track to be notified of all the Python sessions.

English

Day/Time	Topic
4/9 09:00 AM PT	Build your code-first app with Azure AI Agent Service
4/9 03:00 PM PT	Build your code-first app with Azure AI Agent Service
4/10 12:00 PM PT	Transforming business processes with multi-agent AI using Semantic Kernel
4/15 09:00 AM PT	Building Agentic Applications with AutoGen v0.4
4/15 03:00 PM PT	Prototyping AI Agents with GitHub Models
4/16 09:00 AM PT	Building agents with an army of models from the Azure AI model catalog
4/16 12:00 PM PT	Multi-Agent API with LangGraph and Azure Cosmos DB
4/16 03:00 PM PT	Mastering Agentic RAG
4/17 09:00 AM PT	Building smarter Python AI agents with code interpreters
4/17 03:00 PM PT	Agentic Voice Mode Unplugged
4/22 06:00 AM PT	Building a AI Agent with Prompty and Azure AI Foundry
4/22 09:00 AM PT	Real-time Multi-Agent LLM solutions with SignalR, gRPC, and HTTP based on Semantic Kernel
4/22 03:00 PM PT	VoiceRAG: talk to your data
4/23 09:00 AM PT	Building Multi-Agent Apps on top of Azure PostgreSQL
4/23 12:00 PM PT	Agentic RAG with reflection
4/24 09:00 AM PT	Extending AI Agents with Azure Functions
4/24 12:00 PM PT	Build real time voice agents with Azure Communication Services
4/24 03:00 PM PT	Bringing robots to life: Real-time interactive experiences with Azure OpenAI GPT-4o
4/29, 03:00 PM PT	Evaluating Agents

Spanish / Español

Estas transmisiones tratan de Python, pero están en español. Tambien puedes registrar para todas las sesiones en español.

Día/Hora	Tema
4/16 09:00 AM PT	Crea tu aplicación de código con Azure AI Agent Service
4/17 09:00 AM PT	Construyendo agentes utilizando un ejército de modelos con el catálogo de Azure AI Foundry
4/17 12:00 PM PT	Crea aplicaciones de agentes de IA con Semantic Kernel
4/22 12:00 PM PT	Prototipando agentes de IA con GitHub Models
4/23 12:00 PM PT	Comunicación dinámica en agentes grupales
4/23 03:00 PM PT	VoiceRAG: habla con tus datos

Portuguese / Português

Somente uma transmissão está focada em Python, mas você pode se inscrever para todas as sessões em português.

Dia/Horário	Tópico
4/10 12:00 PM PT	Crie um aplicativo com o Azure AI Agent Service

Weekly office hours

To help you with all your questions about building AI agents in Python, we’ll also be holding weekly office hours on the AI Discord server:

Day/Time	Topic/Hosts
Every Thursday, 12:30 PM PT	Python + AI (English)
Every Monday, 03:00 PM PT	Python + AI (Spanish)

We hope to see you at the streams or office hours! If you do have any questions about the hackathon, please reach out to us in the hackathon discussion forum or Discord channel.

The post Build AI agents with Python in #AgentsHack appeared first on Microsoft for Python Developers Blog.

April 05, 2025 12:11 AM UTC

April 04, 2025

TechBeamers Python

Code Without Limits: The Best Online Python Compilers for Every Dev

Explore the top online Python compilers for free. With these, your development environment is always just one browser tab away. Imagine this: You’re sitting in a coffee shop when inspiration strikes. You need to test a Python script immediately, but your laptop is at home. No problem! Whether you’re: These browser-based tools eliminate the friction […]

Source

April 04, 2025 06:31 PM UTC

Python Engineering at Microsoft

Python in Visual Studio Code – April 2025 Release

We’re excited to announce the April 2025 release of the Python, Pylance and Jupyter extensions for Visual Studio Code!

This release includes the following announcements:

Enhanced Python development using Copilot and Notebooks
Improved support for editable installs
Faster and more reliable diagnostic experience (Experimental)
Pylance custom Node.js arguments

If you’re interested, you can check the full list of improvements in our changelogs for the Python, Jupyter and Pylance extensions.

Enhanced Python development using Copilot and Notebooks

The latest improvements to Copilot aim to simplify notebook workflows for Python developers. Sign in to a GitHub account to use Copilot for free in VS Code!

Copilot now supports editing notebooks, using both edit mode and agent mode, so you can effortlessly modify content across multiple cells, insert and delete cells, and adjust cell types—all without interrupting your flow.

VS Code also now supports a new tool for creating Jupyter notebooks using Copilot. This feature plans and creates notebooks based on your query and is supported in all of the various Copilot modes:

Edit mode with chat.edits2.enabled:true enabled.
Agent mode for autonomous peer programming.
Ask mode utilizing the /newNotebook command for quick notebook creation tailored to your project needs.

Lastly, you can now add notebook cell outputs, such as text, errors, and images, directly to chat as context. Use the Add cell output to chat action, available via the triple-dot menu or by right-clicking the output. This lets you reference the output when using ask, edit, or agent mode, making it easier for the language model to understand and assist with your notebook content.

These updates expand Copilot support for Python developers in the Notebook ecosystem enhancing your development workflow no matter the file type.

Improved support for editable installs

Pylance now supports resolving import paths for packages installed in editable mode (pip install -e .) as defined by PEP 660 which enables an improved IntelliSense experience in scenarios such as local development of packages or collaborating on open source projects.

This feature is enabled via setting(python.analysis.enableEditableInstalls:true) and we plan to start rolling it out as the default experience throughout this month. If you experience any issues, please report them at the Pylance GitHub repository.

Faster and more reliable diagnostic experience (Experimental)

In this release, we are rolling out a new update to enhance the accuracy and responsiveness of Pylance’s diagnostics. This update is particularly beneficial in scenarios involving multiple open or recently closed files.

If you do not want to wait for the roll out, you can set setting(python.analysis.usePullDiagnostics:true). If you experience any issues, please report them at the Pylance GitHub repository.

Pylance custom Node.js arguments

You can now pass custom Node.js arguments directly to Node.js with the new setting(python.analysis.nodeArguments) setting, when using setting(python.analysis.nodeExecutable). By default, the setting is configured as "--max-old-space-size=8192". However, you can adjust this value to better suit your needs. For instance, increasing the memory allocation can be helpful when working with large workspaces in Node.js.

Additionally, when setting setting(python.analysis.nodeExecutable) to auto, Pylance now automatically downloads Node.js.

We would also like to extend special thanks to this month’s contributors:

@Sclafus Updated condarc.json in vscode-python#24918
@pheonix-18 Added Python 2.13-dev to test actions in vscode-isort#330
@Riddhi-Thanki Updated default interpreter description in vscode-isort#328
@apollo13 Update minimum required Python version to Python 3.8 in vscode-isort#338
@aparna0522 Updated packages for extension in vscode-isort#332
@archont94 Fixed selecting isort settings from path in vscode-isort#386
@connorads Updated config example in vscode-isort#390
@jicruz96 Do not log traceback if file has skip_file comment in vscode-isort#416
@iloveitaly Added tool path so isort works without bundled version in vscode-isort#417

Try out these new improvements by downloading the Python extension and the Jupyter extension from the Marketplace, or install them directly from the extensions view in Visual Studio Code (Ctrl + Shift + X or ⌘ + ⇧ + X). You can learn more about Python support in Visual Studio Code in the documentation. If you run into any problems or have suggestions, please file an issue on the Python VS Code GitHub page.

The post Python in Visual Studio Code – April 2025 Release appeared first on Microsoft for Python Developers Blog.

April 04, 2025 05:41 PM UTC

Real Python

The Real Python Podcast – Episode #245: GUIs & TUIs: Choosing a User Interface for Your Python Project

What are the current Python graphical user interface libraries? Should you build everything in the terminal and create a text-based user interface instead? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

April 04, 2025 12:00 PM UTC

April 03, 2025

Giampaolo Rodola

Speedup pytest startup

Preface: the migration to pytest

Last year, after 17 years since the inception of the project, I decided to start adopting pytest into psutil (see psutil/#2446). The advantages over unittest are numerous, but the two I cared about most are:

Being able to use base assert statements instead of unittest's self.assert*() APIs.
The excellent pytest-xdist extension, that lets you run tests in parallel, basically for free.

Beyond that, I don't rely on any pytest-specific features in the code, like fixtures or conftest.py. I still organize tests in classes, with each one inheriting from unittest.TestCase. Why?

I like unittest's self.addCleanup too much to give it up (see some usages). I find it superior to fixtures. Less magical and more explicit.
I want users to be able to test their psutil installation in production environments where pytest might not be installed. To accommodate this, I created a minimal "fake" pytest class that emulates essential features like pytest.raises, @pytest.skip etc. (see PR-2456).

But that's a separate topic. What I want to focus on here is one of pytest's most frustrating aspects: slow startup times.

pytest invocation is slow

To measure pytest's startup time, let's run a very simple test where execution time won't significantly affect the results:

$ time python3 -m pytest --no-header psutil/tests/test_misc.py::TestMisc::test_version
============================= test session starts =============================
collected 1 item
psutil/tests/test_misc.py::TestMisc::test_version PASSED
============================== 1 passed in 0.05s ==============================

real    0m0,427s
user    0m0,375s
sys     0m0,051s

0,427s. Almost half of a second. That's excessive for something I frequently execute during development. For comparison, running the same test with unittest:

$ time python3 -m unittest psutil.tests.test_misc.TestMisc.test_version
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK

real    0m0,204s
user    0m0,169s
sys     0m0,035s

0,204 secs. Meaning unittest is roughly twice as fast as pytest. But why?

Where is time being spent?

A significant portion of pytest's overhead comes from import time:

$ time python3 -c "import pytest"
real    0m0,151s
user    0m0,135s
sys     0m0,016s

$ time python3 -c "import unittest"
real    0m0,065s
user    0m0,055s
sys     0m0,010s

There's nothing I can do about that. For the record, psutil import timing is:

$ time python3 -c "import psutil"
real    0m0,056s
user    0m0,050s
sys     0m0,006s

Disable plugin auto loading

After some research, I discovered that pytest automatically loads all plugins installed on the system, even if they aren't used. Here's how to list them (output is cut):

$ pytest --trace-config --collect-only
...
active plugins:
    ...
    setupplan           : ~/.local/lib/python3.12/site-packages/_pytest/setupplan.py
    stepwise            : ~/.local/lib/python3.12/site-packages/_pytest/stepwise.py
    warnings            : ~/.local/lib/python3.12/site-packages/_pytest/warnings.py
    logging             : ~/.local/lib/python3.12/site-packages/_pytest/logging.py
    reports             : ~/.local/lib/python3.12/site-packages/_pytest/reports.py
    python_path         : ~/.local/lib/python3.12/site-packages/_pytest/python_path.py
    unraisableexception : ~/.local/lib/python3.12/site-packages/_pytest/unraisableexception.py
    threadexception     : ~/.local/lib/python3.12/site-packages/_pytest/threadexception.py
    faulthandler        : ~/.local/lib/python3.12/site-packages/_pytest/faulthandler.py
    instafail           : ~/.local/lib/python3.12/site-packages/pytest_instafail.py
    anyio               : ~/.local/lib/python3.12/site-packages/anyio/pytest_plugin.py
    pytest_cov          : ~/.local/lib/python3.12/site-packages/pytest_cov/plugin.py
    subtests            : ~/.local/lib/python3.12/site-packages/pytest_subtests/plugin.py
    xdist               : ~/.local/lib/python3.12/site-packages/xdist/plugin.py
    xdist.looponfail    : ~/.local/lib/python3.12/site-packages/xdist/looponfail.py
    ...

It turns out PYTEST_DISABLE_PLUGIN_AUTOLOAD environment variable can be used to disable them. By running PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 pytest --trace-config --collect-only again I can see that the following plugins disappeared:

anyio
pytest_cov
pytest_instafail
pytest_subtests
xdist
xdist.looponfail

Now let's run the test again by using PYTEST_DISABLE_PLUGIN_AUTOLOAD:

$ time PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python3 -m pytest --no-header psutil/tests/test_misc.py::TestMisc::test_version
============================= test session starts =============================
collected 1 item
psutil/tests/test_misc.py::TestMisc::test_version PASSED
============================== 1 passed in 0.05s ==============================

real    0m0,285s
user    0m0,267s
sys     0m0,040s

We went from 0,427 secs to 0,285 secs, a ~40% improvement. Not bad. We now need to selectively enable only the plugins we actually use, via -p CLI option. Plugins used by psutil are pytest-instafail and pytest-subtests (we'll think about pytest-xdist later):

$ time PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python3 -m pytest -p instafail -p subtests --no-header psutil/tests/test_misc.py::TestMisc::test_version
========================================================= test session starts =========================================================
collected 1 item
psutil/tests/test_misc.py::TestMisc::test_version PASSED
========================================================== 1 passed in 0.05s ==========================================================
real    0m0,320s
user    0m0,283s
sys     0m0,037s

Time went up again, from 0,285 secs to 0,320s. Quite a slowdown, but still better than the initial 0,427s. Now, let's add pytest-xdist to the mix:

$ time PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python3 -m pytest -p instafail -p subtests -p xdist --no-header psutil/tests/test_misc.py::TestMisc::test_version
========================================================= test session starts =========================================================
collected 1 item
psutil/tests/test_misc.py::TestMisc::test_version PASSED
========================================================== 1 passed in 0.05s ==========================================================

real    0m0,369s
user    0m0,286s
sys     0m0,049s

We now went from 0,320s to 0,369s. Not too much, but still it's a pity to pay the price when NOT running tests in parallel.

Handling pytest-xdist

If we disable pytest-xdist psutil tests still run, but we get a warning:

psutil/tests/test_testutils.py:367
  ~/svn/psutil/psutil/tests/test_testutils.py:367: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group(name="serial")

This warning appears for methods that are intended to run serially, those decorated with @pytest.mark.xdist_group(name="serial"). However, since pytest-xdist is now disabled, the decorator no longer exists. To address this, I implemented the following solution in psutil/tests/__init__.py:

import pytest, functools

PYTEST_PARALLEL = "PYTEST_XDIST_WORKER" in os.environ  # True if running parallel tests

if not PYTEST_PARALLEL:
    def fake_xdist_group(*_args, **_kwargs):
        """Mimics `@pytest.mark.xdist_group` decorator. No-op: it just
        calls the test method or return the decorated class."""
        def wrapper(obj):
            @functools.wraps(obj)
            def inner(*args, **kwargs):
                return obj(*args, **kwargs)

            return obj if isinstance(obj, type) else inner

        return wrapper

    pytest.mark.xdist_group = fake_xdist_group  # monkey patch

With this in place the warning disappears when running tests serially. To run tests in parallel, we'll manually enable xdist:

$ python3 -m pytest -p xdist -n auto --dist loadgroup

Disable some default plugins

pytests also loads quite a bunch of plugins by default (see output of pytest --trace-config --collect-only). I tried to disable some of them with:

pytest -p no:junitxml -p no:doctest -p no:nose -p no:pastebin

...but that didn't make much of a difference.

Optimizing test collection time

By default, pytest searches the entire directory for tests, adding unnecessary overhead. In pyproject.toml you can tell pytest where test files are located:

[tool.pytest.ini_options]
testpaths = ["psutil/tests/"]

With this I saved another 0.03 seconds. Before:

$ python3 -m pytest --collect-only
...
======================== 685 tests collected in 0.20s =========================

After:

$ python3 -m pytest --collect-only
...
======================== 685 tests collected in 0.17s =========================

Putting it all together

With these small optimizations, I managed to reduce pytest startup time by ~0.12 seconds, bringing it down from 0.42 seconds. While this improvement is insignificant for full test runs, it somewhat makes a noticeable difference (~28% faster) when repeatedly running individual tests from the command line, which is something I do frequently during development. Final result is visible in PR-2538.

Other links which may be useful

April 03, 2025 10:00 PM UTC

Everyday Superpowers

Why I Finally Embraced Event Sourcing—And Why You Should Too

This is the first entry in a five-part series about event sourcing:

Why I Finally Embraced Event Sourcing—And Why You Should Too
What is event sourcing and why you should care
Preventing painful coupling
Event-driven microservice in a monolith
Get started with event sourcing today

A project I’m working on processes files in multiple phases. To help users track progress, I built a status page that shows details like this:

html

File Name	Pages	Percent complete	Last updated
5466-harbor-4542.pdf	23	33%	two minutes ago
5423-seeds-5675.pdf	35	50%	five minutes ago
9021-lights-3980.pdf	19	100%	30 seconds ago

alignment

normal

After using it for a while, the team had a request: they wanted to see how long each file took to process. That seemed like a useful addition, so I set out to add it.

One way to track processing time is simple: create two new columns in the database called `start_time` and `end_time` and populate them as the documents are being processed. Then, subtract `start_time` from `end_time` to get the duration. If `end_time` doesn’t exist, subtract `start_time` from the current time.

That works well—for new files.

But what about files that have already been processed? How do we estimate their duration?

The Common Problem: Data Loss in Traditional Systems

This is a familiar challenge. Over and over in my career, I’ve seen business requirements change. Maybe our understanding of the project improves, or we discover a better way of doing things. But there’s always a frustrating reality: any new behavior we introduce can only apply going forward. The existing data is locked in its current form.

Why? Because traditional applications lose information.

The database holds the latest version of every row, and when updated, the row overwrites the older information with the new.

The Game-Changer: Event Sourcing

My application doesn’t have this problem.

That’s because I built it using event sourcing—a pattern where instead of just storing the latest state of the system, we store every change as a sequence of events.

With event sourcing, I had data going back to day one. That meant I could calculate the duration for every file submitted to the system.

In just a few minutes, I adjusted some code, ran some tests, and confirmed that I could retroactively compute durations for all past files, even ones that had failed partway through processing.

Then came my favorite moment.

Since my status page updates live via HTMX over a server-sent-events connection, I watched in real-time as durations magically appeared next to every file. The data had always been there, I just added a new way to present it.

And for the first time in my career, I didn’t have to say, “We can do this going forward, but…”

Why I Wish I Had Used Event Sourcing Sooner

I first learned about event sourcing over a decade ago. The concept fascinated me, but I was hesitant to use it in a production system.

Then, after getting laid off last year, I finally had time to experiment with it on side projects. That gave me the confidence to introduce it to a project at work.

And I wish I had done it years ago.

What Is Event Sourcing?

Event sourcing is a way of building applications that never lose data.

Traditional applications update records in place. If a user removes an item from their cart, the application updates the database to reflect the new state. But that means we’ve lost valuable history—we can’t tell what was removed or when.

With event sourcing, every change is stored as an immutable event. Instead of just storing the final cart contents, we store every action. A user’s shopping cart interaction could look like this:

html

Event ID	Cart ID	Event Type	Data	Timestamp
23	1234	CartCreated	{}	2025-01-12T11:01:31
24	1234	ItemAdded	{“product_id”: 2}	2025-01-12T11:02:15
25	1234	ItemAdded	{“product_id”: 5}	2025-01-12T11:05:42
26	1234	ItemRemoved	{“product_id”: 2}	2025-01-12T11:06:59
27	1234	CheckedOut	{}	2025-01-12T11:07:10

alignment

normal

To get the current cart state, we replay the events in order.

This simple shift—from storing state to storing history—changes everything.

“But Isn’t That Slow?”

Surprisingly, no.

Replaying events for a single entity is incredibly fast—it’s just a simple query that retrieves rows in order. I’ve been told that retrieving and replaying hundreds of events[hundreds]{Chances are you won't be building history from hundreds of events. There's a concept called "closing the books" that will keep your event streams small.} is faster than most SQL statements with a join clause.

And when you need to query large amounts of data, like for a dashboard or reporting status for a number of items, event-sourced applications create read models—precomputed views optimized for fast retrieval.

So while a CRUD-based system needs complex queries to piece together data stored across tables, an event-sourced system has the same data ready to go.

Why I Love Event Sourcing

No More “Going Forward” Caveats

The biggest win? When business needs change, I don’t have to tell stakeholders, “We can only do this for new data.”

Instead, I can just replay the history and calculate what they need—even for data that existed before we thought to track it.

Microservices Without the Complexity

At one point, this project I'm working on had five separate event-driven microservices, each serving specific purposes.

After adopting event sourcing in one of those services, I looked to see if we could simplify the project by relocating code to the event-sourced system and have the code subscribe to events.

To my surprise, I realized we could incorporate all of them into one service that was simpler to understand, maintain, and deploy.

Blazing fast views

By incrementally updating read models custom-made for your complicated web pages and API endpoints, those views that used to take time to render can return quickly.

I've seen this approach turn an expensive file-creation-and-download action into a simple static file transfer.

It Just Feels Right

I can’t overstate how satisfying it is to trust that no data is ever lost.

No more digging through logs to reconstruct what happened. No more uncertainty when debugging. The entire history is right there.

Should You Use Event Sourcing?

Absolutely. Yes.

Hear me out. I'm not saying you should go and rewrite your production app. I'm saying that you should use it.

Try it out on a small side project so you know what it's like.

I kept thinking I needed to build a full-blown application and write my own implementation of the event sourcing mechanisms before I would feel comfortable trying it for a job. As a result, it took me over a decade before I even tried it.

Instead, I ask that you be open about the idea of event sourcing, read this series, think about it, try it on as an exercise, and let me know what you think.

I’ve been inspired by Adam Dymitruk and Martin Dilger who both own consulting agencies that use event sourcing in for every project… even those focused on high-frequency trading. They've been operating this way for over a decade and have learned how powerful the pattern is and how to keep it simple. I'll be sharing what I've learned from them over the next few posts.

But for some perspective, after nearly 20 years of writing software and a few years of coaching people on how to write software, the way I write code has changed drastically in this last year.

Check back for the next post, where I'll get more into the practical details of it.

Mike Driscoll

ANN: Spring Python eBook Sale 2025

I am running a Spring sale on all my currently published Python books. You can get 25% off any of my complete books by using this code at checkout: MSON4QP

Learn Python Today!

I have books on the following topics:

Basic Python
Intermediate Python
Python and PDFs
Python and Excel
Image Processing with Python
Python Logging
JupyterLab and Jupyter Notebook
Creating GUIs with wxPython
and more!

Start learning some Python today!

The post ANN: Spring Python eBook Sale 2025 appeared first on Mouse Vs Python.

April 03, 2025 12:11 PM UTC

Python GUIs

Build a Desktop Sticky Notes Application with PySide6 & SQLAlchemy — Create moveable desktop reminders with Python

Do you ever find yourself needing to take a quick note of some information but have nowhere to put it? Then this app is for you! This virtual sticky notes (or Post-it notes) app allows you to keep short text notes quickly from anywhere via the system tray. Create a new note, paste what you need in. It'll stay there until you delete it.

The application is written in PySide6 and the notes are implemented as decoration-less windows, that is windows without any controls. Notes can be dragged around the desktop and edited at will. Text in the notes and note positions are stored in a SQLite database, via SQLAlchemy, with note details and positions being restored on each session.

This is quite a complicated example, but we'll be walking through it slowly step by step. The full source code is available, with working examples at each stage of the development if you get stuck.

Table of Contents

Setting Up the Working Environment

In this tutorial, we'll use the PySide6 library to build the note app's GUI. We'll assume that you have a basic understanding of PySide6 apps.

To learn the basics of PySide6, check out the complete PySide6 Tutorials or my book Create GUI Applications with Python & PySide6

To store the notes between sessions, we will use SQLAlchemy with a SQLite database (a file). Don't worry if you're not familiar with SQLAlchemy, we won't be going deep into that topic & have working examples you can copy.

With that in mind, let's create a virtual environment and install our requirements into it. To do this, you can run the following commands:

macOS
Windows
Linux

sh
$ mkdir notes/
$ cd notes
$ python -m venv venv
$ source venv/bin/activate
(venv)$ pip install pyside6 sqlalchemy

cmd
> mkdir notes/
> cd notes
> python -m venv venv
> venv\Scripts\activate.bat
(venv)> pip install pyside6 sqlalchemy

sh
$ mkdir notes/
$ cd notes
$ python -m venv venv
$ source venv/bin/activate
(venv)$ pip install pyside6 sqlalchemy

With these commands, you create a notes/ folder for storing your project. Inside that folder, you create a new virtual environment, activate it, and install PySide6 and SQLAlchemy from PyPi.

For platform-specific troublshooting, check the Working With Python Virtual Environments tutorial.

Building the Notes GUI

Let's start by building a simple notes UI where we can create, move and close notes on the desktop. We'll deal with persistance later.

The UI for our desktop sticky notes will be a bit strange since there is no central window, all the windows are independent yet look identical (aside from the contents). We also need the app to remain open in the background, using the system tray or toolbar, so we can show/hide the notes again without closing and re-opening the application each time.

We'll start by defining a single note, and then deal with these other issues later. Create a new file named notes.py and add the following outline application to it.

python

import sys

from PySide6.QtWidgets import QApplication, QTextEdit, QVBoxLayout, QWidget

app = QApplication(sys.argv)


class NoteWindow(QWidget):
    def __init__(self):
        super().__init__()
        layout = QVBoxLayout()
        self.text = QTextEdit()
        layout.addWidget(self.text)
        self.setLayout(layout)


note = NoteWindow()
note.show()
app.exec()

In this code we first create a Qt QApplication instance. This needs to be done before creating our widgets. Next we define a simple custom window class NoteWindow by subclassing QWidget. We add a vertical layout to the window, and enter a single QTextEdit widget. We then create an instance of this window object as note and show it by calling .show(). This puts the window on the desktop. Finally, we start up our application by calling app.exec().

You can run this file like any other Pythons script.

python notes.py

When the applicaton launches you'll see the following on your desktop.

Our editable Simple "notes" window on the desktop

If you click in the text editor in the middle, you can enter some text.

Technically this is a note, but we can do better.

Styling our notes

Our note doesn't look anything like a sticky note yet. Let's change that by applying some simple styles to it.

Firstly we can change the colors of the window, textarea and text. In Qt there are multiple ways to do this -- for example, we could override the system palette definition for the window. However, the simplest approach is to use QSS, which is Qt's version of CSS.

python

import sys

from PySide6.QtWidgets import QApplication, QTextEdit, QVBoxLayout, QWidget

app = QApplication(sys.argv)


class NoteWindow(QWidget):
    def __init__(self):
        super().__init__()
        self.setStyleSheet(
            "background: #FFFF99; color: #62622f; border: 0; font-size: 16pt;"
        )
        layout = QVBoxLayout()
        self.text = QTextEdit()
        layout.addWidget(self.text)
        self.setLayout(layout)


note = NoteWindow()
note.show()
app.exec()

In the code above we have set a background color of hex #ffff99 for our note window, and set the text color to hex #62622f a sort of muddy brown. The border:0 removes the frame from the text edit, which otherwise would appear as a line on the bottom of the window. Finally, we set the font size to 16 points, to make the notes easier to read.

If you run the code now you'll see this, much more notely note.

The note with our QSS styles applied The note with the QSS styling applied

Remove Window Decorations

The last thing breaking the illusion of a sticky note on the desktop is the window decorations -- the titlebar and window controls. We can remove these using Qt window flags. We can also use a window flag to make the notes appear on top of other windows. Later we'll handle hiding and showing the notes via a tray application.

python

import sys

from PySide6.QtCore import Qt
from PySide6.QtWidgets import (
    QApplication,
    QTextEdit,
    QVBoxLayout,
    QWidget,
)

app = QApplication(sys.argv)


class NoteWindow(QWidget):
    def __init__(self):
        super().__init__()
        self.setWindowFlags(
            self.windowFlags()
            | Qt.WindowType.FramelessWindowHint
            | Qt.WindowType.WindowStaysOnTopHint
        )
        self.setStyleSheet(
            "background: #FFFF99; color: #62622f; border: 0; font-size: 16pt;"
        )
        layout = QVBoxLayout()

        self.text = QTextEdit()
        layout.addWidget(self.text)
        self.setLayout(layout)


note = NoteWindow()
note.show()
app.exec()

To set window flags, we need to import the Qt flags from the QtCore namespace. Then you can set flags on the window using .setWindowFlags(). Note that since windows have flags already set, and we don't want to replace them all, we get the current flags with .windowFlags() and then add the additional flags to it using boolean OR |. We've added two flags here -- Qt.WindowType.FramelessWindowHint which removes the window decorations, and Qt.WindowType.WindowStaysOnTopHint which keeps the windows on top.

Run this and you'll see a window with the decorations removed.

Note with the window decorations removed

With the window decorations removed you no longer have access to the close button. But you can still close the window using Alt-F4 (Windows) or the application menu (macOS).

While you can close the window, it'd be nicer if there was a button to do it. We can add a custom button using QPushButton and hook this up to the window's .close() method to re-implement this.

python

import sys

from PySide6.QtCore import Qt
from PySide6.QtWidgets import (
    QApplication,
    QHBoxLayout,
    QPushButton,
    QTextEdit,
    QVBoxLayout,
    QWidget,
)

app = QApplication(sys.argv)


class NoteWindow(QWidget):
    def __init__(self):
        super().__init__()
        self.setWindowFlags(
            self.windowFlags()
            | Qt.WindowType.FramelessWindowHint
            | Qt.WindowType.WindowStaysOnTopHint
        )
        self.setStyleSheet(
            "background: #FFFF99; color: #62622f; border: 0; font-size: 16pt;"
        )
        layout = QVBoxLayout()
        # layout.setSpacing(0)

        buttons = QHBoxLayout()
        self.close_btn = QPushButton("×")
        self.close_btn.setStyleSheet(
            "font-weight: bold; font-size: 25px; width: 25px; height: 25px;"
        )
        self.close_btn.setCursor(Qt.CursorShape.PointingHandCursor)
        self.close_btn.clicked.connect(self.close)
        buttons.addStretch()  # Add stretch on left to push button right.
        buttons.addWidget(self.close_btn)
        layout.addLayout(buttons)

        self.text = QTextEdit()
        layout.addWidget(self.text)
        self.setLayout(layout)


note = NoteWindow()
note.show()
app.exec()

Our close button is created using QPushButton with a unicode multiplication symbol (an x) as the label. We set a stylesheet on this button to size the label and button. Then we set a custom cursor on the button to make it clearer that this is a clickable thing that performs an action. Finally, we connect the .clicked signal of the button to the window's close method self.close. The button will close the window.

Later we'll use this button to delete notes.

To add the close button to the top right of the window, we create a horizontal layout with QHBoxLayout. We first add a stretch, then the push button. This has the effect of pushing the button to the right. Finally, we add our buttons layout to the main layout of the note, before the text edit. This puts it at the top of the window.

Run the code now and our note is complete!

The complete note UI with close button

Movable notes

The note looks like a sticky note now, but we can't move it around and there is only one (unless we run the application multiple times concurrently). We'll fix both of those next, starting with the moveability of the notes.

This is fairly straightforward to achieve in PySide because Qt makes the raw mouse events available on all widgets. To implement moving, we can intercept these events and update the position of the window based on the distance the mouse has moved.

To implement this, add the following two methods to the bottom of the NoteWindow class.

python

class NoteWindow(QWidget):
    # ... existing code skipped

    def mousePressEvent(self, e):
        self.previous_pos = e.globalPosition()

    def mouseMoveEvent(self, e):
        delta = e.globalPosition() - self.previous_pos
        self.move(self.x() + delta.x(), self.y() + delta.y())
        self.previous_pos = e.globalPosition()

Clicking and dragging a window involves three actions: the mouse press, the mouse move and the mouse release. We have defined two methods here mousePressEvent and mouseMoveEvent. In mousePressEvent we receive the initial press of the mouse and store the position where the click occurred. This method is only called on the initial press of the mouse when starting to drag the window.

The mouseMoveEvent is called on every subsequent move while the mouse button remains pressed. On each move we take the new mouse position and subtract the previous position to get the delta -- that is, the change in mouse position from the initial press to the current event. Then we move the window by that amount, storing the new previous position after the move.

The effect of this is that ever time the mouseMoveEvent method is called, the window moves by the amount that the mouse has moved since the last call. The window moves -- or is dragged -- by the mouse.

Multiple notes

The note looks like a note, it is now moveable, but there is still only a single note -- not hugely useful! Let's fix that now.

Currently we're creating the NoteWindow when the application starts up, just before we call app.exec(). If we create new notes while the application is running it will need to happen in a function or method, which is triggered somehow. This introduces a new problem, since we need to have some way to store the NoteWindow objects so they aren't automatically deleted (and the window closed) when the function or method exits.

Python automatically deletes objects when they fall out of scope if there aren't any remaining references to them.

We can solve this by storing the NoteWindow objects somewhere. Usually we'd do this on our main window, but in this app there is no main window. There are a few options here, but in this case we're going to use a simple dictionary.

python

import sys

from PySide6.QtCore import Qt
from PySide6.QtWidgets import (
    QApplication,
    QHBoxLayout,
    QPushButton,
    QTextEdit,
    QVBoxLayout,
    QWidget,
)

app = QApplication(sys.argv)

# Store references to the NoteWindow objects in this, keyed by id.
active_notewindows = {}


class NoteWindow(QWidget):
    def __init__(self):
        super().__init__()
        self.setWindowFlags(
            self.windowFlags()
            | Qt.WindowType.FramelessWindowHint
            | Qt.WindowType.WindowStaysOnTopHint
        )
        self.setStyleSheet(
            "background: #FFFF99; color: #62622f; border: 0; font-size: 16pt;"
        )
        layout = QVBoxLayout()

        buttons = QHBoxLayout()
        self.close_btn = QPushButton("×")
        self.close_btn.setStyleSheet(
            "font-weight: bold; font-size: 25px; width: 25px; height: 25px;"
        )
        self.close_btn.clicked.connect(self.close)
        self.close_btn.setCursor(Qt.CursorShape.PointingHandCursor)
        buttons.addStretch()  # Add stretch on left to push button right.
        buttons.addWidget(self.close_btn)
        layout.addLayout(buttons)

        self.text = QTextEdit()
        layout.addWidget(self.text)
        self.setLayout(layout)

        # Store a reference to this note in the
        active_notewindows[id(self)] = self

    def mousePressEvent(self, e):
        self.previous_pos = e.globalPosition()

    def mouseMoveEvent(self, e):
        delta = e.globalPosition() - self.previous_pos
        self.move(self.x() + delta.x(), self.y() + delta.y())
        self.previous_pos = e.globalPosition()


def create_notewindow():
    note = NoteWindow()
    note.show()


create_notewindow()
create_notewindow()
create_notewindow()
create_notewindow()
app.exec()

In this code we've added our active_notewindows dictionary. This holds references to our NoteWindow objects, keyed by id(). Note that this is Python's internal id for this object, so it is consistent and unique. We can use this same id to remove the note. We add each note to this dictionary at the bottom of it's __init__ method.

Next we've implemented a create_notewindow() function which creates an instance of NoteWindow and shows it, just as before. Nothing else is needed, since the note itself handles storing it's references on creation.

Finally, we've added multiple calls to create_notewindow() to create multiple notes.

Multiple notes on the desktop

Adding Notes to the Tray

We can now create multiple notes programatically, but we want to be able to do this from the UI. We could implement this behavior on the notes themselves, but then it wouldn't work if al the notes had been closed or hidden. Instead, we'll create a tray application -- this will show in the system tray on Windows, or on the macOS toolbar. Users can use this to create new notes, and quit the application.

There's quite a lot to this, so we'll step through it in stages.

Update the code, adding the imports shown at the top, and the rest following the definition of create_notewindow.

python

import sys

from PySide6.QtCore import Qt
from PySide6.QtGui import QIcon
from PySide6.QtWidgets import (
    QApplication,
    QHBoxLayout,
    QPushButton,
    QSystemTrayIcon,
    QTextEdit,
    QVBoxLayout,
    QWidget,
)

# ... code hidden up to create_notewindow() definition

create_notewindow()

# Create system tray icon
icon = QIcon("sticky-note.png")

# Create the tray
tray = QSystemTrayIcon()
tray.setIcon(icon)
tray.setVisible(True)


def handle_tray_click(reason):
    # If the tray is left-clicked, create a new note.
    if (
        QSystemTrayIcon.ActivationReason(reason)
        == QSystemTrayIcon.ActivationReason.Trigger
    ):
        create_notewindow()


tray.activated.connect(handle_tray_click)

app.exec()

In this code we've first create an QIcon object passing in the filename of the icon to use. I'm using a sticky note icon from the Fugue icon set by designer Yusuke Kamiyamane. Feel free to use any icon you prefer.

We're using a relative path here. If you don't see the icon, make sure you're running the script from the same folder or provide the path.

The system tray icon is managed through a QSystemTrayIcon object. We set our icon on this, and set the tray icon to visible (so it is not automatically hidden by Windows).

QSystemTrayIcon has a signal activated which fires whenever the icon is activated in some way -- for example, being clicked with the left or right mouse button. We're only interested in a single left click for now -- we'll use the right click for our menu shortly. To handle the left click, we create a handler function which accepts reason (the reason for the activation) and then checks this against QSystemTrayIcon.ActivationReason.Trigger. This is the reason reported when a left click is used.

If the left mouse button has been clicked, we call create_notewindow() to create a new instance of a note.

If you run this example now, you'll see the sticky note in your tray and clicking on it will create a new note on the current desktop! You can create as many notes as you like, and once you close them all the application will close.

The sticky note icon in the tray

This is happening because by default Qt will close an application once all it's windows have closed. This can be disabled, but we need to add another way to quit before we do it, otherwise our app will be unstoppable.

To allow the notes application to be closed from the tray, we need a menu. Sytem tray menus are normally accessible through right-clicking on the icon. To implement that we can set a QMenu as a context menu on the QSystemTrayIcon. The actions in menus in Qt are defined using QAction.

python

import sys

from PySide6.QtCore import Qt
from PySide6.QtGui import QAction, QIcon
from PySide6.QtWidgets import (
    QApplication,
    QHBoxLayout,
    QMenu,
    QPushButton,
    QSystemTrayIcon,
    QTextEdit,
    QVBoxLayout,
    QWidget,
)

# ... code hidden up to handle_tray_click

tray.activated.connect(handle_tray_click)


# Don't automatically close app when the last window is closed.
app.setQuitOnLastWindowClosed(False)

# Create the menu
menu = QMenu()
add_note_action = QAction("Add note")
add_note_action.triggered.connect(create_notewindow)
menu.addAction(add_note_action)

# Add a Quit option to the menu.
quit_action = QAction("Quit")
quit_action.triggered.connect(app.quit)
menu.addAction(quit_action)

# Add the menu to the tray
tray.setContextMenu(menu)


app.exec()

We create the menu using QMenu. Actions are created using QAction passing in the label as a string. This is the text that will be shown for the menu item. The .triggered signal fires when the action is clicked (in a menu, or toolbar) or activated through a keyboard shortcut. Here we've connected the add note action to our create_notewindow function. We've also added an action to quit the application. This is connected to the built-in .quit slot on our QApplication instance.

The menu is set on the tray using .setContextMenu(). In Qt context menus are automatically shown when the user right clicks on the tray.

Finally, we have also disabled the behavior of closing the application when the last window is closed using app.setQuitOnLastWindowClosed(False). Now, once you close all the windows, the application will remain running in the background. You can close it by going to the tray, right-clicking and selecting "Quit".

If you find this annoying while developing, just comment this line out again.

We've had a lot of changes so far, so here is the current complete code.

python

import sys

from PySide6.QtCore import Qt
from PySide6.QtGui import QIcon
from PySide6.QtWidgets import (
    QApplication,
    QHBoxLayout,
    QPushButton,
    QSystemTrayIcon,
    QTextEdit,
    QVBoxLayout,
    QWidget,
)

app = QApplication(sys.argv)

# Store references to the NoteWindow objects in this, keyed by id.
active_notewindows = {}


class NoteWindow(QWidget):
    def __init__(self):
        super().__init__()
        self.setWindowFlags(
            self.windowFlags()
            | Qt.WindowType.FramelessWindowHint
            | Qt.WindowType.WindowStaysOnTopHint
        )
        self.setStyleSheet(
            "background: #FFFF99; color: #62622f; border: 0; font-size: 16pt;"
        )
        layout = QVBoxLayout()

        buttons = QHBoxLayout()
        self.close_btn = QPushButton("×")
        self.close_btn.setStyleSheet(
            "font-weight: bold; font-size: 25px; width: 25px; height: 25px;"
        )
        self.close_btn.clicked.connect(self.close)
        self.close_btn.setCursor(Qt.CursorShape.PointingHandCursor)
        buttons.addStretch()  # Add stretch on left to push button right.
        buttons.addWidget(self.close_btn)
        layout.addLayout(buttons)

        self.text = QTextEdit()
        layout.addWidget(self.text)
        self.setLayout(layout)

        # Store a reference to this note in the active_notewindows
        active_notewindows[id(self)] = self

    def mousePressEvent(self, e):
        self.previous_pos = e.globalPosition()

    def mouseMoveEvent(self, e):
        delta = e.globalPosition() - self.previous_pos
        self.move(self.x() + delta.x(), self.y() + delta.y())
        self.previous_pos = e.globalPosition()


def create_notewindow():
    note = NoteWindow()
    note.show()


create_notewindow()

# Create the icon
icon = QIcon("sticky-note.png")

# Create the tray
tray = QSystemTrayIcon()
tray.setIcon(icon)
tray.setVisible(True)


def handle_tray_click(reason):
    # If the tray is left-clicked, create a new note.
    if (
        QSystemTrayIcon.ActivationReason(reason)
        == QSystemTrayIcon.ActivationReason.Trigger
    ):
        create_notewindow()


tray.activated.connect(handle_tray_click)

app.exec()

If you run this now you will be able to right click the note in the tray to show the menu.

The sticky note icon in the tray showing its context menu

Test the Add note and Quit functionality to make sure they're working.

So, now we have our note UI implemented, the ability to create and remove notes and a persistent tray icon where we can also create notes & close the application. The last piece of the puzzle is persisting the notes between runs of the application -- if we leave a note on the desktop, we want it to still be there if we come back tomorrow. We'll implement that next.

Setting up the Notes database

To be able to store and load notes, we need an underlying data model. For this demo we're using SQLAlchemy as an interface to an SQLite database. This provides an Object-Relational Mapping (ORM) interface, which is a fancy way of saying we can interact with the database through Python objects.

We'll define our database in a separate file, to keep the UI file manageable. So start by creating a new file named database.py in your project folder.

In that file add the imports for SQLAlchemy, and instantiate the Base class for our models.

python

from sqlalchemy import Column, Integer, String, create_engine
from sqlalchemy.orm import declarative_base, sessionmaker

Base = declarative_base()

Next in the same database.py file, define our note database model. This inherits from the Base class we've just created, by calling declarative_base()

python

class Note(Base):
    __tablename__ = "note"
    id = Column(Integer, primary_key=True)
    text = Column(String(1000), nullable=False)
    x = Column(Integer, nullable=False, default=0)
    y = Column(Integer, nullable=False, default=0)

Each note object has 4 properties:

id the unique ID for the given note, used to delete from the database
text the text content of the note
x the x position on the screen
y the y position on the screen

Next we need to create the engine -- in this case, this is our SQLite file, which we're calling notes.db.We can then create the tables (if they don't already exist). Since our Note class registers itself with the Base we can do that by calling create_all on the Base class.

python

engine = create_engine("sqlite:///notes.db")

Base.metadata.create_all(engine)

Save the database.py file and run it

python database.py

After it is complete, if you look in the folder you should see the notes.db. This file contains the table structure for the Note model we defined above.

Finally, we need a session to interact with the database from the UI. Since we only need a single session when the app is running, we can go ahead and create it in this file and then import it into the UI code.

Add the following to database.py

python

# Create a session to handle updates.
Session = sessionmaker(bind=engine)
session = Session()

The final complete code for our database interface is shown below

python

from sqlalchemy import Column, Integer, String, create_engine
from sqlalchemy.orm import declarative_base, sessionmaker

Base = declarative_base()


class Note(Base):
    __tablename__ = "note"
    id = Column(Integer, primary_key=True)
    text = Column(String(1000), nullable=False)
    x = Column(Integer, nullable=False, default=0)
    y = Column(Integer, nullable=False, default=0)


engine = create_engine("sqlite:///notes.db")

Base.metadata.create_all(engine)

Session = sessionmaker(bind=engine)
session = Session()

Now that our data model is defined, and our database created, we can go ahead and interface our Notes model into the UI. This will allow us to load notes at startup (to show existing notes), save notes when they are updated and delete notes when they are removed.

Integrating the Data Model into our UI

Our data model holds the text content and x & y positions of the notes. To keep the active notes and model in sync we need a few things.

Each NoteWindow must have it's own associated instance of the Note object.
New Note objects should be created when creating a new NoteWindow.
The NoteWindow should sync it's initial state to a Note if provided.
Moving & editing a NoteWindow should update the data in the Note.
Changes to Note should be synced to the database.

We can tackle these one by one.

First let's setup our NoteWindow to accept, and store a reference to Note objects if provided, or create a new one if not.

python

import sys

from database import Note
from PySide6.QtCore import Qt
from PySide6.QtGui import QAction, QIcon
from PySide6.QtWidgets import (
    QApplication,
    QHBoxLayout,
    QMenu,
    QPushButton,
    QSystemTrayIcon,
    QTextEdit,
    QVBoxLayout,
    QWidget,
)

app = QApplication(sys.argv)

# Store references to the NoteWindow objects in this, keyed by id.
active_notewindows = {}


class NoteWindow(QWidget):
    def __init__(self, note=None):
        super().__init__()

        # ... add to the bottom of the __init__ method

        if note is None:
            self.note = Note()
        else:
            self.note = note

In this code we've imported the Note object from our database.py file. In the __init__ of our NoteWindow we've added an optional parameter to receive a Note object. If this is None (or nothing provided) a new Note will be created instead. The passed, or created note, is then stored on the NoteWindow so we can use it later.

This Note object is still not being loaded, updated, or persisted to the database. So let's implement that next. We add two methods, load() and save() to our NoteWindow to handle the loading and saving of data.

python

from database import Note, session

# ... skipped other imports, unchanged.

class NoteWindow(QWidget):
    def __init__(self, note=None):
        super().__init__()

        # ... modify the close_btn handler to use delete.
        self.close_btn.clicked.connect(self.delete)


        # ... rest of the code hidden.

        # If no note is provided, create one.
        if note is None:
            self.note = Note()
            self.save()
        else:
            self.note = note
            self.load()

    # ... add the following to the end of the class definition.

    def load(self):
        self.move(self.note.x, self.note.y)
        self.text.setText(self.note.text)

    def save(self):
        self.note.x = self.x()
        self.note.y = self.y()
        self.note.text = self.text.toPlainText()
        # Write the data to the database, adding the Note object to the
        # current session and committing the changes.
        session.add(self.note)
        session.commit()

    def delete(self):
        session.delete(self.note)
        session.commit()
        del active_notewindows[id(self)]
        self.close()

The load() method takes the x and y position from the Note object stored in self.note and updates the NoteWindow position and content to match. The save() method takes the NoteWindow position and content and sets that onto the Note object. It then adds the note to the current database session and commits the changes

Each commit starts a new session. Adding the Note to the session is indicating that we want it's changes persisted.

The delete() method handles deletion of the current note. This involves 3 things:

passing the Note object to session.delete to remove it from the database,
deleting the reference to our window from the active_notewindows (so the object will be tidied up)
calling .close() to hide the window immediately.

Usually (2) will cause the object to be cleaned up, and that will close the window indirectly. But that may be delayed, which would mean sometimes the close button doesn't seem to work straight away. We call .close() to make it immediate.

We need to modify the close_btn.clicked signal to point to our delete method.

Next we've added a load() call to the __init__ when a Note object is passed. We also call .save() for newly created notes to persist them immediately, so our delete handler will work before editing.

Finally, we need to handle saving the note whenever it changes. We have two ways that the note can change -- when it's moved, or when it's edited. For the first we could do this on each mouse move, but it's a bit redundant. We only care where the note ends up while dragging -- that is, where it is when the mouse is released. We can get this through the mouseReleased method.

python

from database import Note, session

# ... skipped other imports, unchanged.

class NoteWindow(QWidget):

    # ... add the mouseReleaseEvent to the events on the NoteWindow.

    def mousePressEvent(self, e):
        self.previous_pos = e.globalPosition()

    def mouseMoveEvent(self, e):
        delta = e.globalPosition() - self.previous_pos
        self.move(self.x() + delta.x(), self.y() + delta.y())
        self.previous_pos = e.globalPosition()

    def mouseReleaseEvent(self, e):
        self.save()

    # ... the load and save methods are under here, unchanged.

That's all there is to it: when the mouse button is released, we save the current content and position by calling .save().

You might be wondering why we don't just save the position at this point? Usually it's better to implement a single load & save (persist/restore) handler that can be called for all situations. It avoids needing implementations for each case.

There have been a lot of partial code changes in this section, so here is the complete current code.

python

import sys

from database import Note, session
from PySide6.QtCore import Qt
from PySide6.QtGui import QAction, QIcon
from PySide6.QtWidgets import (
    QApplication,
    QHBoxLayout,
    QMenu,
    QPushButton,
    QSystemTrayIcon,
    QTextEdit,
    QVBoxLayout,
    QWidget,
)

app = QApplication(sys.argv)

# Store references to the NoteWindow objects in this, keyed by id.
active_notewindows = {}


class NoteWindow(QWidget):
    def __init__(self, note=None):
        super().__init__()

        self.setWindowFlags(
            self.windowFlags()
            | Qt.WindowType.FramelessWindowHint
            | Qt.WindowType.WindowStaysOnTopHint
        )
        self.setStyleSheet(
            "background: #FFFF99; color: #62622f; border: 0; font-size: 16pt;"
        )
        layout = QVBoxLayout()

        buttons = QHBoxLayout()
        self.close_btn = QPushButton("×")
        self.close_btn.setStyleSheet(
            "font-weight: bold; font-size: 25px; width: 25px; height: 25px;"
        )
        self.close_btn.clicked.connect(self.delete)
        self.close_btn.setCursor(Qt.CursorShape.PointingHandCursor)
        buttons.addStretch()  # Add stretch on left to push button right.
        buttons.addWidget(self.close_btn)
        layout.addLayout(buttons)

        self.text = QTextEdit()
        layout.addWidget(self.text)
        self.setLayout(layout)

        self.text.textChanged.connect(self.save)

        # Store a reference to this note in the active_notewindows
        active_notewindows[id(self)] = self

        # If no note is provided, create one.
        if note is None:
            self.note = Note()
            self.save()
        else:
            self.note = note
            self.load()

    def mousePressEvent(self, e):
        self.previous_pos = e.globalPosition()

    def mouseMoveEvent(self, e):
        delta = e.globalPosition() - self.previous_pos
        self.move(self.x() + delta.x(), self.y() + delta.y())
        self.previous_pos = e.globalPosition()

    def mouseReleaseEvent(self, e):
        self.save()

    def load(self):
        self.move(self.note.x, self.note.y)
        self.text.setText(self.note.text)

    def save(self):
        self.note.x = self.x()
        self.note.y = self.y()
        self.note.text = self.text.toPlainText()
        # Write the data to the database, adding the Note object to the
        # current session and committing the changes.
        session.add(self.note)
        session.commit()

    def delete(self):
        session.delete(self.note)
        session.commit()
        del active_notewindows[id(self)]
        self.close()


def create_notewindow():
    note = NoteWindow()
    note.show()


create_notewindow()

# Create the icon
icon = QIcon("sticky-note.png")

# Create the tray
tray = QSystemTrayIcon()
tray.setIcon(icon)
tray.setVisible(True)


def handle_tray_click(reason):
    # If the tray is left-clicked, create a new note.
    if (
        QSystemTrayIcon.ActivationReason(reason)
        == QSystemTrayIcon.ActivationReason.Trigger
    ):
        create_notewindow()


tray.activated.connect(handle_tray_click)


# Don't automatically close app when the last window is closed.
app.setQuitOnLastWindowClosed(False)

# Create the menu
menu = QMenu()
# Add the Add Note option to the menu.
add_note_action = QAction("Add note")
add_note_action.triggered.connect(create_notewindow)
menu.addAction(add_note_action)

# Add a Quit option to the menu.
quit_action = QAction("Quit")
quit_action.triggered.connect(app.quit)
menu.addAction(quit_action)
# Add the menu to the tray
tray.setContextMenu(menu)


app.exec()

If you run the application at this point it will be persisting data to the database as you edit it.

If you want to look at the contents of the SQLite database I can recommend DB Browser for SQLite. It's open source & free.

The note data persisted to the SQLite database

Starting up

So our notes are being created, added to the database, updated and deleted. The last piece of the puzzle is restoring the previous state at start up.

We already have all the bits in place for this, we just need to handle the startup itself. To recreate the notes we can query the database to get a list of Note objects and then iterate through this, creating new NoteWindow instances (using our create_notewindow function).

python

def create_notewindow(note=None):
    note = NoteWindow(note)
    note.show()


existing_notes = session.query(Note).all()

if existing_notes:
    for note in existing_notes:
        create_notewindow(note)
else:
    create_notewindow()

First we've modified the create_notewindow function to accept an (optional) Note object which is passed through to the created NoteWindow.

Using the session we query session.query(Note).all() to get all the Note objects. If there any, we iterate them creating them. If not, we create a single note with no associated Note object (this will be created inside the NoteWindow).

That's it! The full final code is shown below:

python

import sys

from database import Note, session
from PySide6.QtCore import Qt
from PySide6.QtGui import QAction, QIcon
from PySide6.QtWidgets import (
    QApplication,
    QHBoxLayout,
    QMenu,
    QPushButton,
    QSystemTrayIcon,
    QTextEdit,
    QVBoxLayout,
    QWidget,
)

app = QApplication(sys.argv)

# Store references to the NoteWindow objects in this, keyed by id.
active_notewindows = {}


class NoteWindow(QWidget):
    def __init__(self, note=None):
        super().__init__()

        self.setWindowFlags(
            self.windowFlags()
            | Qt.WindowType.FramelessWindowHint
            | Qt.WindowType.WindowStaysOnTopHint
        )
        self.setStyleSheet(
            "background: #FFFF99; color: #62622f; border: 0; font-size: 16pt;"
        )
        layout = QVBoxLayout()

        buttons = QHBoxLayout()
        self.close_btn = QPushButton("×")
        self.close_btn.setStyleSheet(
            "font-weight: bold; font-size: 25px; width: 25px; height: 25px;"
        )
        self.close_btn.clicked.connect(self.delete)
        self.close_btn.setCursor(Qt.CursorShape.PointingHandCursor)
        buttons.addStretch()  # Add stretch on left to push button right.
        buttons.addWidget(self.close_btn)
        layout.addLayout(buttons)

        self.text = QTextEdit()
        layout.addWidget(self.text)
        self.setLayout(layout)

        self.text.textChanged.connect(self.save)

        # Store a reference to this note in the active_notewindows
        active_notewindows[id(self)] = self

        # If no note is provided, create one.
        if note is None:
            self.note = Note()
            self.save()
        else:
            self.note = note
            self.load()

    def mousePressEvent(self, e):
        self.previous_pos = e.globalPosition()

    def mouseMoveEvent(self, e):
        delta = e.globalPosition() - self.previous_pos
        self.move(self.x() + delta.x(), self.y() + delta.y())
        self.previous_pos = e.globalPosition()

    def mouseReleaseEvent(self, e):
        self.save()

    def load(self):
        self.move(self.note.x, self.note.y)
        self.text.setText(self.note.text)

    def save(self):
        self.note.x = self.x()
        self.note.y = self.y()
        self.note.text = self.text.toPlainText()
        # Write the data to the database, adding the Note object to the
        # current session and committing the changes.
        session.add(self.note)
        session.commit()

    def delete(self):
        session.delete(self.note)
        session.commit()
        del active_notewindows[id(self)]
        self.close()


def create_notewindow(note=None):
    note = NoteWindow(note)
    note.show()


existing_notes = session.query(Note).all()

if existing_notes:
    for note in existing_notes:
        create_notewindow(note)
else:
    create_notewindow()

# Create the icon
icon = QIcon("sticky-note.png")

# Create the tray
tray = QSystemTrayIcon()
tray.setIcon(icon)
tray.setVisible(True)


def handle_tray_click(reason):
    # If the tray is left-clicked, create a new note.
    if (
        QSystemTrayIcon.ActivationReason(reason)
        == QSystemTrayIcon.ActivationReason.Trigger
    ):
        create_notewindow()


tray.activated.connect(handle_tray_click)


# Don't automatically close app when the last window is closed.
app.setQuitOnLastWindowClosed(False)

# Create the menu
menu = QMenu()
# Add the Add Note option to the menu.
add_note_action = QAction("Add note")
add_note_action.triggered.connect(create_notewindow)
menu.addAction(add_note_action)

# Add a Quit option to the menu.
quit_action = QAction("Quit")
quit_action.triggered.connect(app.quit)
menu.addAction(quit_action)
# Add the menu to the tray
tray.setContextMenu(menu)


app.exec()

If you run the app now, you can create new notes as before, but when you exit (using the Quit option from the tray) and restart, the previous notes will reappear. If you close the notes, they will be deleted. On startup, if there are no notes in the database an initial note will be created for you.

Conclusion

That's it! We have a fully functional desktop sticky note application, which you can use to keep simple bits of text until you need them again. We've learnt how to build an application up step by step from a basic outline window. We've added basic styles using QSS and used window flags to control the appearance of notes on the desktop. We've also seen how to create a system tray application, adding context menus and default behaviours (via a left mouse click). Finally, we've created a simple data model using SQLAlchemy and hooked that into our UI to persist the UI state between runs of the applications.

Try and extend this example further, for example:

Add multicolor notes, using a Python list of hex colors and random.choice to select a new color each time a note is created. Can you persist this in the database too?
Add an option in the tray to show/hide all notes. Remember we have all the NoteWindow objects in active_notewindows. You can show and hide windows in Qt using .show() and .hide().

Think about some additional features you'd like or expect to see in a desktop notes application and see if you can add them yourself!

April 03, 2025 12:01 PM UTC

Seth Michael Larson

Nintendo Switch 2: DRM, expensive, and GameCube

So the Switch 2 got announced in a Nintendo Direct yesterday. The event itself was essentially an unending series of incredible new information about the Switch 2 console and games. Here are my mixed thoughts, especially about things that weren't included in the live stream.

New physical cartridges that don't work offline

I saw on Mastodon that there's a new cartridge coming for the Switch 2 called a "Game-Key Card". The existence of this cartridge is disappointing to me, it's essentially the worst of both worlds for both physical and digital downloads. This cartridge type combines the now more expensive medium (physical) with the medium that removes user rights, preservation, and long-term utility (digital).

Presumably the new "Game Key" cartridges only contain the equivalent of an activation and decryption key along with a link to a download server for installing the game. What this means in practice is the following:

You can no longer sell your physical cartridges.
You can no longer lend your physical cartridges, you need to use Nintendo's new game sharing service which will one day be shut down.
You can no longer buy physical cartridges without first verifying that they had never been installed previously. This might be impossible to do depending on the UI of the Switch 2 prior to installation.
You need to purchase more or larger microSD cards as now every game you purchase needs to be saved to your Switch instead of being stored in the cartridge.
If you don't download and install your game before game servers are taken down then you won't be able to use your purchase.

Despite all the above downsides you still need to have your cartridge plugged into the Switch for the game to work, presumably because the content is encrypted even when on your microSD card. So you don't even get the biggest upsides of digital content: portability and convenience.

I will be curious to see how often this type of game cartridge is used. Apparently a similar mechanism was already happening with existing Switch 1 cartridges. Some publishers would provide the bare-minimum software in a Switch 1 cartridge and then rely on the Nintendo Switch Game Update service to install the full contents of the game.

This new game cartridge type is effectively making this approach to distribution blessed by Nintendo. In a way this is better because the game will be labeled correctly that the contents of the game require an internet connection to run completely, but I suspect it will also mean that this technique will be more common amongst publishers.

All-in-all, this means that when Nintendo shuts down Nintendo Switch online in ~20 years and your Switch console or microSD card are damaged you will lose access to your collection. There is no legal way to produce a backup of your games that incorporate encryption like the Switch cartridges thanks to the DMCA.

Switch 2 might be expensive?

The Switch 2 itself costs $450 USD with a single-game bundle cost of $500 USD. Inflation-adjusted the Switch 1 cost $390 USD in today's money back in 2017, so $450 is a 15% cost increase on launch. I am less concerned about this price difference given it's a one-time purchase, the Switch 2 is a backwards compatible console, and that the Switch family has proven itself to be a valuable entertainment investment.

The very first game announced for the Switch 2 is Mario Kart World, an open-world party kart game. This game is going to cost $80 USD for a physical copy of the game and $70 USD for a digital-only copy. This is the first Nintendo title to cost $80 USD and the first Nintendo title to have a price difference between physical and digital editions. Lots of implications for this...

The difference in cost between physical and digital makes sense to me. Creating something physical in the world is not free, there is some associated cost. However, the video game preservationist in me is frustrated that there's a continued march down the path of not being able to actually own the content you purchase.

As a physical game collector I now need to decide how much my principles are worth, probably more than $10 but for games which are sold as "Game-Key Cards" this price is absolutely not worth it. I do not recommend buying Game-Key Cards, just go digital if you really want the game and this is the only physical cartridge option.

If there were more protections for making legal backups of digital content then none of this would be an issue.

As far as the actual prices, I am hoping that Mario Kart, likely one of the best games for every Nintendo console, is an exception rather than a rule for this price. Mario Kart 8 Deluxe sold 67M units, 10M more than the second-best selling Switch game. Knowing what I know now about the Switch I would pay $80 for Mario Kart 8 Deluxe! I wonder what customers are going to think though now that the first Nintendo Switch 2 game is $20 more than usual.

I worry that other publishers, and even Nintendo themselves, are going to try to push the envelope on game prices. It's never made much sense why every game that's not "indie" needs to be $60 USD, so I hope that consumers feel that way too and don't fall for publishers pushing games that are not "Mario Kart"-levels of quality for even more ludicrous prices beyond the industry-standard $60 USD.

Maybe higher prices will also make the job of game journalists and reviewers even more important and fewer people pre-order games, I think this would be a good development if game prices increase.

Switch 2 requires microSD Express cards

If you bought a bunch of large microSD cards because they'll be useful "eventually", then you'll be a bit disappointed by this one. The Switch 2 requires microSD Express cards. I personally don't own any microSD Express cards but have a handful of 256GB+ microSD cards without "Express".

Doing a quick price check on Amazon shows that a 256GB San-Disk microSD card without "Express" (150MB/s reads) costs $20 USD and with "Express" (880MB/s reads) costs $60 USD. So you might be buying a brand new and more expensive microSD card for your Switch 2 just to store all the games.

This is another part of the price of the Switch 2 to consider when purchasing.

"Single-Pak" / "Download Play" is back with GameShare

What's old is new again: the Switch 2 supports playing a single copy of a game across multiple consoles with a new feature called "GameShare". This feature is similar to the DS and 3DS "Download Play" and the Game Boy Advance "Single-Pak" play modes. Even better: Switch 1 consoles are able to "receive" games being hosted by the Switch 2 (but are unable to "host" the games).

Despite being a "handheld" console, the Switch 1 didn't originally support such a feature. This meant that the Switch was the first handheld that lacked this feature since the Game Boy Color from 1998.

Switch 2 will be the console of GameCube?

The announcement contained a ton of information about GameCube games for the Switch 2. GameCube games are coming to Nintendo Switch Online, in particular I'm excited for F-Zero GX and Pokémon XD: Gale of Darkness which I don't own myself but have always wanted to try.

There's also a new GameCube controller for the Switch that actually uses the classic indigo color, definitely need to get my hands on one.

Finally, they teased Kirby Air Riders which appears to be the sequel to Kirby Air Ride on the GameCube. This game was a favorite of mine and my brother, especially the "City Trial" mode. With Sakurai returning as the director I'm not too worried about the game being a hit, but I do hope they keep a "City Trial"-esque mode instead of making the game all about racing.

April 03, 2025 12:00 AM UTC

April 02, 2025

Real Python

How to Strip Characters From a Python String

By default, Python’s .strip() method removes whitespace characters from both ends of a string. To remove different characters, you can pass a string as an argument that specifies a set of characters to remove. The .strip() method is useful for tasks like cleaning user input, standardizing filenames, and preparing data for storage.

By the end of this tutorial, you’ll understand that:

The .strip() method removes leading and trailing whitespace but doesn’t remove whitespace from the middle of a string.
You can use .strip() to remove specified characters from both ends of the string by providing these characters as an argument.
With the related methods .lstrip() and .rstrip(), you can remove characters from one side of the string only.
All three methods, .strip(), .lstrip(), and .rstrip(), remove character sets, not sequences.
You can use .removeprefix() and .removesuffix() to strip character sequences from the start or end of a string.

In this tutorial, you’ll explore the nuances of .strip() and other Python string methods that allow you to strip parts of a string. You’ll also learn about common pitfalls and read about practical real-world scenarios, such as cleaning datasets and standardizing user input. To get the most out of this tutorial, you should have a basic understanding of Python strings and character data.

Get Your Code: Click here to download the free sample code that shows you how to strip characters from a Python string.

Take the Quiz: Test your knowledge with our interactive “How to Strip Characters From a Python String” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

How to Strip Characters From a Python String

In this quiz, you'll test your understanding of Python's .strip(), .lstrip(), and .rstrip() methods, as well as .removeprefix() and .removesuffix(). These methods are useful for tasks like cleaning user input, standardizing filenames, and preparing data for storage.

How to Use Python’s `.strip()` Method to Remove Whitespace From Strings

Python’s .strip() method provides a quick and reliable way to remove unwanted spaces, tabs, and newline characters from both the beginning and end of a string. This makes it useful for tasks like:

Validating user input, such as trimming spaces from email addresses, usernames, and other user-provided data.
Cleaning messy text gathered through web scraping or other sources.
Preparing data for storage to ensure uniformity before saving text to a database.
Standardizing logs by removing unwanted spaces.

If you don’t provide any arguments to the method, then .strip() removes all leading and trailing whitespace characters, leaving any whitespace within the string untouched:

Python
      
>>> original_string = "   Hello, World!   "
>>> original_string.strip()
'Hello, World!'
Copied!

When you call .strip() on a string object, Python removes the leading and trailing spaces while keeping the spaces between words unchanged, like in "Hello," and "World!". This can be a great way to clean up text data without affecting the content itself.

However, whitespace isn’t just about spaces—it also includes common characters such as newlines (\n) and tabs (\t). These often appear when you’re dealing with multi-line strings or reading data from files. The default invocation of .strip() effectively removes them as well:

Python
      
        
      
    
>>> text = """\n\t  This is a messy multi-line string.
...
...        \t    """
>>> text.strip()
'This is a messy multi-line string.'
Copied!

Here, .strip() removes all leading and trailing whitespace characters, including newlines and tabs, leaving only the text content. After having cleaned your strings using .strip(), they’re in better condition for displaying or further processing the text. This can be especially useful when you’re dealing with structured data, such as logs or CSV files, where you need to process many strings in a row.

At this point, you’ve learned how .strip() handles whitespace removal. But what if you need to strip specific characters, not just whitespace? In the next section, you’ll see how you can use this method to remove any unwanted characters from the start and end of a string.

Remove Specific Characters With `.strip()`

Sometimes, you need to remove specific characters other than whitespace. For example, when your text is delimited by unwanted symbols, or when you have to handle text that’s plagued by formatting issues. You can use .strip() to remove specific characters by passing these characters as an argument to the method:

Python Syntax
      
    
cleaned_string = original_string.strip(chars=None)
Copied!

Here, chars is a string argument that you can pass to .strip(). If you don’t pass it, then it defaults to None, which means the method will remove whitespace characters.

Instead, you can pass a string value that contains all the characters needing removal from both ends of the target string. Note that .strip() doesn’t treat the argument as a prefix or suffix, but rather as a set of individual characters to strip. In the rest of this section, you’ll explore use cases of passing specific characters to .strip() for cleaning the beginning and end of a string.

The .strip() method is useful when you want to remove punctuation marks, specific symbols, or other unwanted characters. For example, in sentiment analysis tasks, you may need to remove question marks or exclamation marks from text data:

Python
      
>>> review = "!!This product is incredible!!!"
>>> review.strip("!")
'This product is incredible'
Copied!

Since you pass "!" as an argument, .strip() removes all exclamation marks from both ends of the string while leaving the text content intact. Keep in mind that .strip() removes all occurrences of the specified characters at once, not just the first one it encounters.

You can also use .strip() to remove multiple specified characters from both ends of a string. For example, some of the product reviews you’re dealing with may be in Spanish and use a combination of exclamation marks and inverted exclamation marks:

Read the full article at https://realpython.com/python-strip/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

April 02, 2025 02:00 PM UTC

Django Weblog

Django 5.2 released

The Django team is happy to announce the release of Django 5.2.

The release notes showcase a composite of new features. A few highlights are:

All models are automatically imported in the shell by default.
Django now supports composite primary keys! The new django.db.models.CompositePrimaryKey allows tables to be created with a primary key consisting of multiple fields.
Overriding a BoundField got a lot easier: this can now be set on a form, field or project level.

You can get Django 5.2 from our downloads page or from the Python Package Index. The PGP key ID used for this release is: 3955B19851EA96EF

With the release of Django 5.2, Django 5.1 has reached the end of mainstream support. The final minor bug fix release, 5.1.8, which was also a security release, was issued today. Django 5.1 will receive security and data loss fixes until December 2025. All users are encouraged to upgrade before then to continue receiving fixes for security issues.

Django 5.0 has reached the end of extended support. The final security release, 5.0.14, was issued today. All Django 5.0 users are encouraged to upgrade to Django 5.1 or later.

See the downloads page for a table of supported versions and the future release schedule.

April 02, 2025 10:16 AM UTC

Django security releases issued: 5.1.8 and 5.0.14

In accordance with our security release policy, the Django team is issuing releases for Django 5.1.8 and Django 5.0.14. These releases address the security issues detailed below. We encourage all users of Django to upgrade as soon as possible.

CVE-2025-27556: Potential denial-of-service vulnerability in `LoginView`, `LogoutView`, and `set_language()` on Windows

Python's NFKC normalization is slow on Windows. As a consequence, django.contrib.auth.views.LoginView, django.contrib.auth.views.LogoutView, and django.views.i18n.set_language were subject to a potential denial-of-service attack via certain inputs with a very large number of Unicode characters.

Thanks to sw0rd1ight for the report.

This issue has severity "moderate" according to the Django security policy.

Affected supported versions

Django main
Django 5.2 (currently at release candidate status)
Django 5.1
Django 5.0

Resolution

Patches to resolve the issue have been applied to Django's main, 5.2 (currently at release candidate status), 5.1, and 5.0 branches. The patches may be obtained from the following changesets.

CVE-2025-27556: Potential denial-of-service vulnerability in `LoginView`, `LogoutView`, and `set_language()` on Windows

On the main branch
On the 5.2 branch
On the 5.1 branch
On the 5.0 branch

The following releases have been issued

Django 5.1.8 (download Django 5.1.8 | 5.1.8 checksums)
Django 5.0.14 (download Django 5.0.14 | 5.0.14 checksums)

The PGP key ID used for this release is : 3955B19851EA96EF

General notes regarding security reporting

As always, we ask that potential security issues be reported via private email to security@djangoproject.com, and not via Django's Trac instance, nor via the Django Forum. Please see our security policies for further information.

April 02, 2025 09:37 AM UTC

Python GUIs

Getting Started with Streamlit — Build your first Streamlit app and explore some basic features

Streamlit is an open-source Python library that makes it easy to create and share custom web apps for machine learning and data science. In this tutorial we'll take a first look at Streamlit, installing it, getting it set up and building a simple app.

Table of Contents

Installing Streamlit

Because Streamlit is a third-party library, we need to install it on our system before using it. Streamlit can be easily installed using pip. Open your terminal (Mac/Linux) or Command Prompt (Windows) and type the following command:

bash

pip install streamlit

This command will download and install Streamlit and its dependencies. Once the installation is complete, we can create a simple Streamlit app.

Open your editor and create a file named app.py. This file will be our main Python file to write and edit the Streamlit app. In order to make sure that Streamlit is running on our system, let us import it and run the app.

python

import streamlit as st

To run the app, open the terminal in the same directory and enter the following commands

bash

streamlit run app.py

This command will open a new tab in your default browser. It will be empty right now, but this is where we'll be building the interface of our app.

Add title, headings, and paragraphs

Streamlit allows you to create clean and structured web apps with ease, making it perfect for data visualization and interactive dashboards. One of the key features that make Streamlit user-friendly is its ability to format text with titles, headings, and paragraphs. This tutorial will guide you through how to add these elements to your Streamlit app.

Adding Titles

Titles in Streamlit are added using the st.title() function. This function displays the text in a large, bold font, ideal for the heading of your app.

python

import streamlit as st

st.title("This is the title of our app")

Save the changes and refresh the browser tab to see the changes. This will create a large, centered title at the top of your app.

The streamlit application open in your browser

Adding Headings

Streamlit provides several levels of headings to structure your content, similar to HTML's <h1> to <h6> tags. You can use st.header() and st.subheader() for the primary and secondary sections, respectively.

python

import streamlit as st

st.title("This is the title of our app")
st.header("This is a Header")
st.subheader("This is a Subheader")

In this code, we use st.header() to create the prominent heading, ideal for section titles. Then we call st.subheader() to create a slightly smaller heading, suitable for subsections under a header.

Save the changes and refresh the browser. It will create the following changes to our app.

Subheaders added through Streamlit

Adding Paragraphs

To add regular text or paragraphs, use the st.write() function. This function can handle text, Markdown, and even complex objects like data frames.

python

import streamlit as st

st.title("This is the title of our app")
st.header("This is a Header")
st.subheader("This is a Subheader")
st.write("You can write text here and it will appear as a paragraph.")

Save the change and refresh the browser tab.

Paragraph text added through Streamlit

Adding different kinds of buttons to the Streamlit app

Buttons are fundamental interactive elements in any web application. Streamlit provides simple yet versatile options for adding buttons to your app, enabling users to trigger actions, navigate between sections, or submit data. This tutorial will guide you through adding different kinds of buttons to your Streamlit app and how to handle user interactions.

Basic Button

The simplest button in Streamlit is created using the st.button() function. It generates a clickable button that can trigger specific actions.

python

import streamlit as st

st.title("This is the title of our app")

st.button("Click Me")

A button in your Streamlit UI

Notice that a small button is shown in our app. Right now, it is a static button which mean nothing will happen when we click the button. To make it interactive, we have to use conditional statements. When the button is clicked in Streamlit app, it returns a True value. So, we can use this in our conditional statement.

python

import streamlit as st

st.title("This is the title of our app")

button = st.button("Click Me")
if button: # button is True if clicked
    st.write("You clicked the button")

We create the button by calling st.button() passing in the label as a string "Click Me". If the button is clicked in the browser, the value of button will be True and the if branch will be executed: outputting the message to the UI.

A button with clickable behavior

Download Button

You can create a download button using st.download_button(), which allows users to download files directly from your app.

python

import streamlit as st

st.title("This is the title of our app")

text_file_content = "This is a sample text file. This content will be downloaded as a text file."

st.download_button(
    label="Download Text File",
    data=text_file_content,
    file_name="sample.txt",
    mime="text/plain"
)

In this code we use st.download_button() to creates a button that, when clicked, lets users download a file. The parameters of the download button are:

label: The text on the button.
data: The content to be downloaded.
file_name: The name of the file.
mime: The MIME type of the file (e.g., text/plain).

This gives the following update UI:

A download button

Radio Buttons for Options

Radio buttons allow users to select one option from a set of choices. Streamlit provides this functionality using st.radio().

python

import streamlit as st

st.title("This is the title of our app")

choice = st.radio("Choose an option:", ["Option 1", "Option 2", "Option 3"])

if choice == "Option 1":
    st.write("You selected Option 1")
elif choice == "Option 2":
    st.write("You selected Option 2")
else:
    st.write("You selected Option 3")

In this case we used st.radio() to create a set of radio buttons. The selected option is stored in the variable choice, which you can use to control the app's behavior.

This will give the following result:

Radio buttons in your Streamlit UI

Adding slider

A slider in Streamlit is a UI element that allows users to select a value by moving a handle along a track. This is particularly useful for adjusting parameters in data visualizations, setting thresholds, or selecting ranges for filtering data.

Streamlit provides st.slider() which you can use to add sliders to your app. This function supports both single-value and range sliders.

A single value slider allows users to select a single value within a specified range. Here’s how to add a simple slider to your Streamlit app:

python

import streamlit as st

st.title("This is the title of our app")

age = st.slider("Select your age:", 0, 100, 25)

st.write(f"Your age is: {age}")

Here we've used st.slider() to add a slider to your app. The first argument is the label for the slider. The next two arguments define the minimum and maximum values, while the the last argument is the default value the slider starts at.

A simple slider in your Streamlit UI

Streamlit also allows you to create a range slider, where users can select an upper and lower bound of a range. This is useful for filtering where you want to select some data within the given range. You can add a range slider to your application as follows:

python

import streamlit as st

st.title("This is the title of our app")

start, end = st.slider("Select a range of values:", 0, 100, (20, 80))

st.write(f"Selected range: {start} to {end}")

A range slider in your Streamlit UI

Here, st.slider() is used to create a range slider by passing a tuple (20, 80) as the default value. The tuple represents the initial start and end values of the slider range.

When you run this app, the slider will allow users to select a range between 0 and 100, starting with a default range from 20 to 80. The selected range is then displayed on the app. The initial and returned tuple represent the selected range in the slider.

Don't confuse this with a Python range! Unlike a Python range, they are inclusive: that is, if you select 80 as the upper bound, then 80 will be returned (not 79).

Dropdown menus are a powerful and versatile UI component that allows users to select an option from a predefined list. Streamlit makes it easy to add dropdown menus to your web app, providing an intuitive way for users to interact with your data or application.

Streamlit provides a straightforward way to add dropdown menus through the st.selectbox() function. This function not only adds a dropdown to your app but also allows you to capture the selected value for further processing. Let’s start with a simple example where users can choose their favorite fruit from a list:

python

import streamlit as st

st.title("This is the title of our app")

fruit = st.selectbox("Select your favorite fruit:", ["Apple", "Banana", "Orange", "Grapes", "Mango"])

st.write(f"You selected: {fruit}")

A dropdown box in your Streamlit UI

In the above code we used st.selectbox() to creates a dropdown menu. The first argument is the label for the dropdown and the second argument is the list of options users can choose from.

A sidebar is an additional panel that appears on the left side of the app. It can be used to house interactive widgets like sliders, buttons, and dropdowns, or to display information that should be easily accessible throughout the app.

This allows the main part of the app to focus on displaying results, visualizations, or other content, while the sidebar handles user inputs and navigation.

Streamlit makes it easy to add a sidebar using the st.sidebar attribute, which you can use to place any widget or content into the sidebar.

To add a sidebar to your Streamlit app, you can use the st.sidebar attribute followed by the widget you want to add. Here’s a basic example of adding a sidebar with a simple slider.

python

import streamlit as st

st.title("This is the title of our app")

age = st.sidebar.slider("Select your age:", 0, 100, 25)

st.write(f"Your age is: {age}")

This will produce the following output:

Adding a sidebar to your Streamlit UI

st.sidebar.slider() is a function that adds a slider widget to the sidebar instead of the main page. The rest of the code works just like a regular slider.

You can add multiple widgets to the sidebar, allowing users to control various aspects of your app from one convenient location. Here’s an example with a dropdown menu, a slider, and a button.

python

import streamlit as st

st.title("This is the title of our app")

color = st.sidebar.selectbox("Select a color:", ["Red", "Green", "Blue"])

# Add a slider to the sidebar
level = st.sidebar.slider("Select the intensity level:", 0, 100, 50)

# Add a button to the sidebar
if st.sidebar.button("Apply Settings"):
    st.write(f"Settings applied: Color={color}, Level={level}")

The updated UI is shown below:

Multiple widgets in the sidebar of your Streamlit UI

In this code we used st.sidebar.selectbox() to add a dropdown menu to the sidebar, st.sidebar.slider() to add a slider to the sidebar and finally st.sidebar.button() to add a button to the sidebar. The the action associated with the button click is displayed on the main page.

Creating a simple web app using Streamlit

Now, we will combine all the basic concepts that we have learned about Streamlit and put them together to create a simple streamlit web app.

In this example we'll being using the Iris dataset. This data is widely available, for example from this repository. Download the csv file into the same folder as your Streamlit app and then we can load it using pandas.

Using this dataset we're going to build a data exploration dashboard with the following features.

Dataset Display showing the full Iris dataset.
Sidebar Controls** to allow users to select a feature and filter the data based on that feature using sliders.
Filtered Data Display to show the filtered dataset based on the user's input.
Basic Statistics like mean, median, standard deviation, calculated live based on the selected feature in the filtered dataset.

The UI is simple, but shows some of the neat features of Streamlit.

python

import streamlit as st
import pandas as pd

# Load the Iris dataset
df = pd.read_csv('iris.csv')

# Set the title of the app
st.title("Iris Dataset Explorer")

# Display the entire dataframe
st.write("### Full Iris Dataset")
st.dataframe(df)

# Sidebar configuration
st.sidebar.header("Filter Options")

# Feature selection
feature = st.sidebar.selectbox("Select a feature to filter by:", df.columns[:-1])

# Range selection based on the selected feature
min_value = float(df[feature].min())
max_value = float(df[feature].max())

range_slider = st.sidebar.slider(f"Select range of {feature}:", min_value, max_value, (min_value, max_value))

# Filter the dataframe based on the selected range
filtered_df = df[(df[feature] >= range_slider[0]) & (df[feature] <= range_slider[1])]

# Display the filtered dataset
st.write(f"### Filtered Iris Dataset by {feature} between {range_slider[0]} and {range_slider[1]}")
st.dataframe(filtered_df)

# Display basic statistics for the filtered data
st.write(f"### Statistics for {feature}")
st.write(filtered_df[feature].describe())

The iris.csv path is relative, so will only work if you run the script from the same folder. If you want to run it from elsewhere (a parent folder) you will need to modify the path.

Below is the final UI, showing the sidebar on the left and the full & filtered Iris dataset in the middle panel. Change the feature and adjust the parameter to filter the data.

Data filtering demo using Pandas & Streamlit

This simple Streamlit app provides an easy way to explore the Iris dataset. It demonstrates how you can use sidebars, dropdown menus, and sliders to create an interactive and user-friendly data exploration tool.

You can take this simple app and adapt it for other data sets or expand it with additional features, such as advanced filtering or data manipulation options.

April 02, 2025 06:00 AM UTC

April 01, 2025

Test and Code

Python 3.14 won't repeat with pytest-repeat

pytest-repeat is a pytest plugin that makes it easy to repeat a single test, or multiple tests, a specific number of times.
Unfortunately, it doesn't seem to work with Python 3.14, even though there is no rational reason why it shouldn't work.

Links:

Sponsored by:

The Complete pytest course is now a bundle, with each part available separately.
- pytest Primary Power teaches the super powers of pytest that you need to learn to use pytest effectively.
- Using pytest with Projects has lots of "when you need it" sections like debugging failed tests, mocking, testing strategy, and CI
- Then pytest Booster Rockets can help with advanced parametrization and building plugins.
Whether you need to get started with pytest today, or want to power up your pytest skills, PythonTest has a course for you.

★ Support this podcast on Patreon ★ pytest-repeat is a pytest plugin that makes it easy to repeat a single test, or multiple tests, a specific number of times. Unfortunately, it doesn't seem to work with Python 3.14, even though there is no rational reason why it shouldn't work.Links:<ul><li><a href="https://github.com/pytest-dev/pytest-repeat">pytest-repeat</a></li><li><a href="https://www.youtube.com/watch?v=wgxBHuUOmjA">Guido van Rossum returns as Python's BDFL</a></li></ul> Sponsored by: <ul><li><a href="https://file+.vscode-resource.vscode-cdn.net/Users/brianokken/projects/test_and_code_notes/new_ad.md">The Complete pytest course</a> is now a bundle, with each part available separately.<ul><li><a href="https://courses.pythontest.com/pytest-primary-power">pytest Primary Power</a> teaches the super powers of pytest that you need to learn to use pytest effectively.</li><li><a href="https://courses.pythontest.com/using-pytest-with-projects">Using pytest with Projects</a> has lots of "when you need it" sections like debugging failed tests, mocking, testing strategy, and CI</li><li>Then <a href="https://courses.pythontest.com/pytest-booster-rockets">pytest Booster Rockets</a> can help with advanced parametrization and building plugins.</li></ul></li><li>Whether you need to get started with pytest today, or want to power up your pytest skills, <a href="https://courses.pythontest.com">PythonTest</a> has a course for you.</li></ul> <a href="https://www.patreon.com/c/testpodcast" rel="payment" title="★ Support this podcast on Patreon ★">★ Support this podcast on Patreon ★</a>

April 01, 2025 10:20 PM UTC

PyCoder’s Weekly

Issue #675: Optimization, DuckDB, Outliers, and More (April 1, 2025)

#675 – APRIL 1, 2025
View in Browser »

An April Fool’s free issue. All content was curated before April 1st and is guranteed to be April Fool’s free.

Optimizing With Generators, Expressions, & Efficiency

Python generators provide an elegant mechanism for handling iteration, particularly for large datasets where traditional approaches may be memory-intensive. Unlike standard functions that compute and return all values at once, generators produce values on demand through the yield statement, enabling efficient memory usage and creating new possibilities for data processing workflows.
PYBITES • Shared by Bob Belderbos

Introducing DuckDB

In this showcase tutorial, you’ll be introduced to a library that allows you to use a database in your code. DuckDB provides an efficient relational database that supports many features you may already be familiar with from more traditional relational database systems.
REAL PYTHON

Quiz: Introducing DuckDB

REAL PYTHON

Learn AI In 5 Minutes A Day

Everyone talks about AI, but no one has the time to learn it. So, we found the simplest way to learn AI as quickly as possible: The Rundown AI. It’s the most trusted AI newsletter, with 1M+ readers and exclusives with AI leaders like Mark Zuckerberg, Demis Hassibis, Mustafa Suleyman, and more →
THE RUNDOWN AI sponsor

Outlier Detection With Python

Have you ever wondered why certain data points stand out so dramatically? They might hold the key to everything from fraud detection to groundbreaking discoveries. This week Talk Python to Me interviews Brett Kennedy on outlier detection.
TALK PYTHON podcast

Articles & Tutorials

What Can You Do With Python?

In this video course, you’ll find a set of guidelines that will help you start applying your Python skills to solve real-world problems. By the end, you’ll be able to answer the question, “What can you do with Python?”
REAL PYTHON course

Python Code Quality: Best Practices and Tools

In this tutorial, you’ll learn about code quality and the key factors that make Python code high-quality. You’ll explore effective strategies, powerful tools, and best practices to elevate your code to the next level.
REAL PYTHON

Optimizing Django by Not Being Silly

Although the post is nominally about a tool being used with Django, the root problem being debugged is about handling substrings. Doing it badly can be a real performance bottleneck, learn how to avoid the problem.
MAX BERNSTEIN

Share Python Scripts Like a Pro

Sharing single-file Python scripts with external dependencies is now easy thanks to uv and PEP 723, which enable embedding dependency metadata directly within script files.
DAVE JOHNSON • Shared by Dave Johnson

PEP 781: Make TYPE_CHECKING a Built-in Constant

This PEP proposes adding a new built-in variable, TYPE_CHECKING, which is True when the code is being analyzed by a static type checker, and False during normal runtime.
PYTHON.ORG

Checking User Permissions in Django Templates

Not all actions in your web project are for all users. This post shows you how to check what a user’s permissions are from within the HTML template being rendered.
TIM KAMANIN

Checking Whether Iterables Are Equal in Python

“You can check whether iterables contain the same elements in Python with equality checks, type conversions, sets, Counter, or looping helpers.”
TREY HUNNER

Understanding Numpy’s `einsum`

Einstein notation lets you evaluate operations on multi-dimensional arrays. NumPy has this built-in. This post shows you how to use it.
ELI BENDERSKY

Building a Real-Time Dashboard With FastAPI and Svelte

Learn how to build a real-time analytics dashboard using FastAPI and Svelte with server-sent events.
AMIR TADRISI • Shared by Michael Herman

Building Accessible Web Forms in Django

A step by step reference to building accessible web forms in Django.
VALENTINO GAGLIARDI

Quiz: Python Code Quality: Best Practices and Tools

REAL PYTHON

Projects & Code

python-docx: Create and Modify Word Documents

GITHUB.COM/PYTHON-OPENXML

Cirron: Trace System Calls That Python Executes

GITHUB.COM/S7NFO

pythonx: Python Interpreter Embedded in Elixir

GITHUB.COM/LIVEBOOK-DEV

docs: Collaborative Note Taking, Wiki and Docs Platform

GITHUB.COM/SUITENUMERIQUE

py-bugger: Practice Debugging, Intentionally Introduce Bugs

GITHUB.COM/EHMATTHES

Events

Weekly Real Python Office Hours Q&A (Virtual)

April 2, 2025
REALPYTHON.COM

Canberra Python Meetup

April 3, 2025
MEETUP.COM

Sydney Python User Group (SyPy)

April 3, 2025
SYPY.ORG

Python Communities

April 5 to April 6, 2025
NOKIDBEHIND.ORG

PyDelhi User Group Meetup

April 5, 2025
MEETUP.COM

Python Conference Austria 2025

April 6 to April 8, 2025
PYCON.ORG

Happy Pythoning!
This was PyCoder’s Weekly Issue #675.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

April 01, 2025 07:30 PM UTC

Real Python

Building a Code Image Generator With Python

If you’re active on social media, then you know that images and videos are popular forms of content. As a programmer, you mainly work with text, so sharing the content that you create on a daily basis may not seem intuitive. That’s where a code image generator comes in handy!

A code image generator allows you to turn your code snippets into visually appealing images, so you can share your work without worrying about formatting issues, syntax highlighting inconsistencies, or character count limits.

In this step-by-step video course, you’ll learn how to:

Set up and run a Flask project
Connect and style Jinja templates
Use Playwright to create images
Beautify code with Pygments
Leverage sessions to save browser states
Enhance the user experience with JavaScript

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

April 01, 2025 02:00 PM UTC

Mike Driscoll

Textual – How to Add Widgets to a Container

Textual is an excellent Python package for creating beautiful user interfaces in your terminal. By default, Textual will arrange your widgets starting at the top of the screen and appending them in a vertically oriented stack. Each GUI or TUI toolkit provides a way to lay out your widgets. Textual is no different in this respect. They use an object called a container.

You can use containers to create the following types of layouts:

Vertical layout
Horizontal layout
Grid layout
and more!

You will be learning how to use all three of these types of layouts. You will also learn how to add more widgets at runtime.

Let’s get started!

Creating a Vertical Layout

The default orientation in Textual is to arrange widgets vertically. You don’t even need to use a CSS file to apply this orientation.

But what does a vertical layout mean anyway? A vertical layout is when you add widgets to your application vertically, from top to bottom. Here is an illustration of what that might look like:

Textual vertical layout illustration

Adding widgets to a Textual application will lay out the widgets similarly to the image above. If you want to see that for yourself, then open up your Python editor and create a new file named `vertical.py`.

Then enter the following code into your new script:

# vertical.py

from textual.app import App, ComposeResult
from textual.widgets import Button


class VerticalApp(App):

    def compose(self) -> ComposeResult:
        yield Button("OK")
        yield Button("Cancel")
        yield Button("Go!")


if __name__ == "__main__":
    app = VerticalApp()
    app.run()

Now open up a terminal and run your code. When you do so, you will see three buttons onscreen, with the topmost being your “OK” button and the bottom being the “Go!” button.

Here is a screenshot of the application to give you an idea of what it looks like:

Textual vertical (no CSS)
You can change the widget size, color, and more using each widget’s styles attribute, but using CSS is simpler. Let’s update the code above to use a vertical.tcss file:

# verical_css.py

from textual.app import App, ComposeResult
from textual.widgets import Button


class VerticalApp(App):
    CSS_PATH = "vertical.tcss"

    def compose(self) -> ComposeResult:
        yield Button("OK")
        yield Button("Cancel")
        yield Button("Go!")


if __name__ == "__main__":
    app = VerticalApp()
    app.run()

Now that you are referring to a CSS file, you should go ahead and write one. If you don’t, you will get an error when you attempt to run the code that says the CSS file could not be found.

Go ahead and open your favorite text editor or use your Python editor to create a file named `vertical.tcss`. Then enter the following code:

Screen {
    layout: vertical;
}

Button {
    width: 100%;
    color: yellow;
    background: red;
}

You do not need the Screen portion of the CSS since that is technically taken care of automatically by Textual. Remember, Screen is the default widget when you launch an application. However, it is always good to be explicit so you understand what is happening. If you want the output to look exactly like the previous example, you can delete this CSS’s Button portion and try running the code that way.

If you decide to include the Button portion of the CSS, you will make all of the Button widgets 100% wide, which means they will all stretch across the entire width of the screen. The CSS also defines the button text to be yellow and the buttons themselves to have a read background color.

When you run this code, you will see something like the following:

Textual vertical layout with CSS

That’s a fun way to change your vertically oriented widget layout. But what happens if you set the height of the Button widgets to 50%? Well, you have three widgets. Three times 50 will be 150%, which is greater than what can be shown all at once. Textual will add a scrollbar if you add widgets that go off-screen.

Try adding that setting to your CSS and re-run the code. You should see something like the following:

Textual with vertical layout CSS and height at 50%

You should spend a few moments trying out various width and height sizes. Remember, you don’t have to use percentages. You can also use Textual’s other unit types.

Note: All style attributes can be adjusted at runtime, which means you can modify the layout at runtime, too. Use this wisely so as not to confuse the user!

When you finish experimenting, you will be ready to learn how horizontal layouts work!

Horizontal Layout

Laying widgets out horizontally, left-to-right, requires a little more work than laying them out vertically. But the change is still pretty minor, and in many ways, it affects only one line in the CSS file.

But before you change the CSS, you will want to update your Python code to point to the new CSS file. Open your Python editor and copy the previous example to a new file. Save it with the same horizontal.py and update the CSS_PATH to point to a new CSS file named horizontal.tcss:

# horizontal.py

from textual.app import App, ComposeResult
from textual.widgets import Button


class HorizontalApp(App):
    CSS_PATH = "horizontal.tcss"

    def compose(self) -> ComposeResult:
        yield Button("OK")
        yield Button("Cancel")
        yield Button("Go!")


if __name__ == "__main__":
    app = HorizontalApp()
    app.run()

Yes, this code is almost the same as the previous example, except the CSS_PATH variable. That’s okay. The point is to show you how you can change the layout.

Create your horizontal.tcss file in a Python or text editor to make a horizontally oriented layout. Then enter the following CSS:

Screen {
    layout: horizontal;
}

Button {
    height: 100%;
    color: yellow;
    background: red;
    border: solid green;
}

The CSS above added a border to the buttons to make them stand out a bit more. Depending on the terminal, the widgets appear to blend together more when arranged horizontally. You can add space around the widgets by setting the margin style, though.

When you run this code, you should see something like the following:

Textual horizontal layout with CSS

When using a horizontal layout, the horizontal scrollbar will not automatically appear if the widgets do not fit the screen. If you want to have a horizontal scrollbar, then you will need to set overflow-x: auto;, like in the following CSS:

Screen {
    layout: horizontal;
    overflow-x: auto;
}

Button {
    height: 100%;
    color: yellow;
    background: red;
    border: solid green;
}

Now, set the widgets’ width to greater than 33% so that the scrollbar will appear. Spend some time experimenting, and you’ll soon figure it out!

Layouts with Containers

The Textual package has several utility containers you can use to lay out your widgets. You are most likely to use Vertical, Horizontal, or Grid containers. You can also combine the containers to create more complex layouts.

Here is a full list of the containers included with Textual at the time of writing:

Center
Container
Horizontal
HorizontalScroll
Middle
ScrollableContainer
Vertical
VerticalScroll

You will most likely use the Center, Middle, Horizontal, and Vertical containers the most.

Practicing is the best learning method, especially when laying out user interfaces. You can start your container journey by opening your Python editor and creating a new file called horizontal_container.py. Then enter the following code:

# horizontal_container.py

from textual.app import App, ComposeResult
from textual.widgets import Button
from textual.containers import Horizontal


class HorizontalApp(App):

    def compose(self) -> ComposeResult:
        yield Horizontal(
            Button("OK"),
            Button("Cancel"),
            Button("Go!"),
        )


if __name__ == "__main__":
    app = HorizontalApp()
    app.run()

You import the Horizontal container from textual.containers. The main contents of a container is its widgets. You reuse the widgets from the previous example here. Pay attention and note that you do not need to use yield inside the container. You can simply add the widget instances instead.

When you run this code, you will see something like this:

Textual horizontal container

What will happen if you use your horizontal.tcss file with this code? Try adding it to the code above and re-run your example.

The result will look familiar:

Textual horizontal container plus CSS

The real benefit using containers comes when you nest them. You’ll find out about that concept next!

Nesting Containers

Nesting containers allows you to combine horizontally and vertically oriented widgets, resulting in rows and columns of widgets. This design pattern can create some pretty nice layouts.

To start, create a new file called nested_containers.py in your Python editor. Then add this code to it:

# nested_containers.py

from textual.app import App, ComposeResult
from textual.widgets import Button
from textual.containers import Horizontal, Vertical


class NestedApp(App):

    def compose(self) -> ComposeResult:
        yield Vertical(
            Horizontal(
                Button("One"),
                Button("Two"),
                classes="row",
            ),
            Horizontal(
                Button("Three"),
                Button("Four"),
                classes="row",
            ),
        )


if __name__ == "__main__":
    app = NestedApp()
    app.run()

Your code above has a single Vertical container with two Horizontal containers inside. You can think of the Horizontal containers as “rows”. You can see that you set the classes parameters to “row” to identify them. Each row contains two Button widgets.

When you run this code, you will see something like this:

Textual nested containers

This example doesn’t use any CSS. You should do that! Update the code to include a CSS file called nested.tcss, like the code below:

# nested_containers.py

from textual.app import App, ComposeResult
from textual.widgets import Button
from textual.containers import Horizontal, Vertical


class NestedApp(App):
    CSS_PATH = "nested.tcss"

    def compose(self) -> ComposeResult:
        yield Vertical(
            Horizontal(
                Button("One"),
                Button("Two"),
                classes="row",
            ),
            Horizontal(
                Button("Three"),
                Button("Four"),
                classes="row",
            ),
        )


if __name__ == "__main__":
    app = NestedApp()
    app.run()

Then, create the nested.tcss file. You will be putting the following CSS rules into it:

Button {
    content-align: center middle;
    background: green;
    border: yellow;
    height: 1fr;
    width: 1fr;
}

Here, you set various rules for the Button widgets to follow. You want the buttons to be green with a yellow border. You also set the width and height to 1fr, which causes the buttons to expand to fit all the horizontal and vertical space.

When you run this version of your code, you can see that the user interface has changed significantly:

Textual nested containers

Nice! You should spend some time adjusting the style rules and seeing how to change these layouts.

Wrapping Up

Learning how to create layouts is a fundamental skill that you will need to master to be able to create engaging, intuitive user interfaces. Fortunately, Textual gives you enough tools that you can create your user interfaces fairly easily. No; you don’t get a What-you-see-is-what-you-get (WYSIWYG) tool as you do with some GUI toolkits, such as QT Creator. But you do get live-coding with CSS, and since most of your user interface layouts are controlled there, tweaking the user interface is so nicer.

Want to Learn More Textual?

This tutorial is based on a chapter from my latest book, Creating TUI Applications with Textual and Python.

Creating TUI Applications with Textual and Python

You will learn everything you need to know about Textual from this book. You will also create TEN small applications to apply what you learn. Check it out today!

The post Textual – How to Add Widgets to a Container appeared first on Mouse Vs Python.

April 01, 2025 01:44 PM UTC

Zero to Mastery

[March 2025] Python Monthly Newsletter 🐍

64th issue of Andrei Neagoie's must-read monthly Python Newsletter: Django Got Forked, The Science of Troubleshooting, Python 3.13 TLDR, and much more. Read the full newsletter to get up-to-date with everything you need to know from last month.

April 01, 2025 10:00 AM UTC

eGenix.com

Python Meeting Düsseldorf - 2025-04-09

The following text is in German, since we're announcing a regional user group meeting in Düsseldorf, Germany.

Ankündigung

Das nächste Python Meeting Düsseldorf findet an folgendem Termin statt:

09.04.2025, 18:00 Uhr
Raum 1, 2.OG im Bürgerhaus Stadtteilzentrum Bilk
Düsseldorfer Arcaden, Bachstr. 145, 40217 Düsseldorf

Programm

Bereits angemeldete Vorträge

Fabian Preiß:
How Not to Use Python Match Cases
Ulf Morys:
n8n - Erster Einblick in low code workflow automation... mit und ohne Python
Johannes Dröge:
Distributed, Data-driven Applications with Python and ZebraStream
Daniel Schmitz:
Dataframevalidierung mit pandera

Weitere Vorträge können gerne noch angemeldet werden. Bei Interesse, bitte unter info@pyddf.de melden.

Startzeit und Ort

Wir treffen uns um 18:00 Uhr im Bürgerhaus in den Düsseldorfer Arcaden.

Das Bürgerhaus teilt sich den Eingang mit dem Schwimmbad und befindet sich an der Seite der Tiefgarageneinfahrt der Düsseldorfer Arcaden.

Über dem Eingang steht ein großes "Schwimm’ in Bilk" Logo. Hinter der Tür direkt links zu den zwei Aufzügen, dann in den 2. Stock hochfahren. Der Eingang zum Raum 1 liegt direkt links, wenn man aus dem Aufzug kommt.

>>> Eingang in Google Street View

⚠️ Wichtig: Bitte nur dann anmelden, wenn ihr absolut sicher seid, dass ihr auch kommt. Angesichts der begrenzten Anzahl Plätze, haben wir kein Verständnis für kurzfristige Absagen oder No-Shows.

Einleitung

Das Python Meeting Düsseldorf ist eine regelmäßige Veranstaltung in Düsseldorf, die sich an Python Begeisterte aus der Region wendet.

Einen guten Überblick über die Vorträge bietet unser PyDDF YouTube-Kanal, auf dem wir Videos der Vorträge nach den Meetings veröffentlichen.

Veranstaltet wird das Meeting von der eGenix.com GmbH, Langenfeld, in Zusammenarbeit mit Clark Consulting & Research, Düsseldorf:

Format

Das Python Meeting Düsseldorf nutzt eine Mischung aus (Lightning) Talks und offener Diskussion.

Vorträge können vorher angemeldet werden, oder auch spontan während des Treffens eingebracht werden. Ein Beamer mit HDMI und FullHD Auflösung steht zur Verfügung.

(Lightning) Talk Anmeldung bitte formlos per EMail an info@pyddf.de

Kostenbeteiligung

Das Python Meeting Düsseldorf wird von Python Nutzern für Python Nutzer veranstaltet.

Da Tagungsraum, Beamer, Internet und Getränke Kosten produzieren, bitten wir die Teilnehmer um einen Beitrag in Höhe von EUR 10,00 inkl. 19% Mwst. Schüler und Studenten zahlen EUR 5,00 inkl. 19% Mwst.

Wir möchten alle Teilnehmer bitten, den Betrag in bar mitzubringen.

Anmeldung

Da wir nur 25 Personen in dem angemieteten Raum empfangen können, möchten wir bitten, sich vorher anzumelden.

Meeting Anmeldung bitte per Meetup

Weitere Informationen

Weitere Informationen finden Sie auf der Webseite des Meetings:

https://pyddf.de/

Viel Spaß !

Marc-Andre Lemburg, eGenix.com

April 01, 2025 08:00 AM UTC

Tryton News

Newsletter April 2025

Last month we focused on fixing bugs, improving the behaviour of things, speeding-up performance issues - building on the changes from our last release. We also added some new features which we would like to introduce to you in this newsletter.

For an in depth overview of the Tryton issues please take a look at our issue tracker or see the issues and merge requests filtered by label.

Changes for the User

CRM, Sales, Purchases and Projects

Now we notify the user when trying to add a duplicate contact mechanism.

Add quotation validity date on sale and purchase quotations.
On sale we compute the validity date when it goes to state quotation and display the validity date in the report. On purchase we set the date directly.

It is a common practice among other things to answer a complain by giving a promotion coupon to the customer. Now the user can create a coupon from the sale complain as an action.

We now use the actual quantity of a sale line when executing a sale complaint,
when the product is already selected.

Now we add an relate to open all products of sales, to be able to check all the sold products (for quantity or price).

We simplify the coupon number management and added a menu entry for promotion coupon numbers.

Now we display a coupon form on promotions and we remove the name field on promotion coupons.

Accounting, Invoicing and Payments

Now we allow to download all pending SEPA messages in a single message report.

We now replace the maturity date on account move by a combined payable/receivable date field which contains a given maturity date and if it is empty, falls back to the effective date. This provides a better chronology of the move lines.

On account move we now replace the post_number by the number-field. The original functionality of the number field, - delivering a sequential number for account moves in draft state, - is replaced by the account move id.

We now add some common payment terms:

Upon Receipt
Net 10, 15, 30, 60 days
Net 30, 60 days End of Month
End of Month
End of Month Following

Now we display an optional company-field on the payment and group list.

We now add tax identifiers to the company. A company may have two tax identifiers, one used for transactions inland and another used abroad. Now it is possible to select the company tax identifier based on rules.

Now we make the deposit-field optional on party list view.

We now use the statement date to compute the start balance instead of always using the last end balance.

Now we make entries in analytic accounting read-only depending on their origin state.

We now allow to delete landed costs only if they are cancelled.

Now we add the company field optionally to the SEPA mandate list.

Stock, Production and Shipments

We now add the concept of product place also to the inventory line, because some users may want to see the place when doing inventory so they know where to count the products exactly.

Now we display the available quantity when searching in a stock move
and if the product is already selected:

We now ship packages of internal shipments with transit.

Now we do no longer force to fill an incoterm when shipping inside Europe.

User Interface

In the web client now we scroll to the first selected element in the tree view, when switching from form view.

Now we add a color widget to the form view.

Also we now add an icon of type color, to display the color visually in a tree view. We extend the image type to add color which just displays an image filled with color.

Now we deactivate the open-button of the One2Many widget, if there is no form view.

In the desktop client we now include the version number on the new version available message.

System Data and Configuration

In the web user form we now use the same structure as in user form.

Now we make the product attribute names unique. Because the name of the attributes are used as keys of a fields.Dict.

We now add the Yapese currency Rai.

Now we order the incoming documents by their descending ID, with the most recent documents on top.

New Documentation

Now we add an example of a payment term with multiple deltas.

We now reworked the web_sh‎op_shopify module documentation.

New Releases

We released bug fixes for the currently maintained long term support series
7.0 and 6.0, and for the penultimate series 7.4 and 7.2.

Changes for Implementers and Developers

Now we raise UserErrors from the database exceptions, to log more information on data and integrity errors.

In the desktop client we now remove the usage of GenericTreeModel, the last remaining part of pygtkcompat in Tryton.

We now make it easy to extend the Sendcloud sender address with a pattern.

Now we set a default value for all fields of a wizard state view.
If the client does not display a field of a state view, the value of this field on the instance record is not a defined attribute. So we need to access it using getattr with a default value, but in theory this can happen for any state in any record as user can extend any view.

We now store the last version series to which the database was updated in ir.configuration. With this information, the list of databases is filtered to the same client series. The remote access to a database is restricted to databases available in the list. We now also return the series instead of the version for remote call.

Authors: @dave @pokoli @udono spoiler

1 post - 1 participant

Read full topic

April 01, 2025 06:00 AM UTC

Wingware

Wing Python IDE 11 Early Access - March 27, 2025

Wing 11 is now available as an early access release, with improved AI assisted development, support for the uv package manager, improved Python code analysis, improved custom key binding assignment user interface, improved diff/merge, a new preference to auto-save files when Wing loses the application focus, updated German, French and Russian localizations (partly using AI), a new experimental AI-driven Spanish localization, and other bug fixes and minor improvements.

You can participate in the early access program simply by downloading the early access releases. We ask only that you keep your feedback and bug reports private by submitting them through Wing's Help menu or by emailing us at support@wingware.com.

Downloads

IMPORTANT Be sure to Check for Updates from Wing's Help menu after installing so that you have the latest hot fixes.

Wing Pro 11.0.0.1

Wing Personal 11.0.0.1

Wing 101 11.0.0.1

Wing 10 and earlier versions are not affected by installation of Wing 11 and may be installed and used independently. However, project files for Wing 10 and earlier are converted when opened by Wing 11 and should be saved under a new name, since Wing 11 projects cannot be opened by older versions of Wing.

New in Wing 11

Improved AI Assisted Development

Wing 11 improves the user interface for AI assisted development by introducing two separate tools AI Coder and AI Chat. AI Coder can be used to write, redesign, or extend code in the current editor. AI Chat can be used to ask about code or iterate in creating a design or new code without directly modifying the code in an editor.

This release also improves setting up AI request context, so that both automatically and manually selected and described context items may be paired with an AI request. AI request contexts can now be stored, optionally so they are shared by all projects, and may be used independently with different AI features.

AI requests can now also be stored in the current project or shared with all projects, and Wing comes preconfigured with a set of commonly used requests. In addition to changing code in the current editor, stored requests may create a new untitled file or run instead in AI Chat. Wing 11 also introduces options for changing code within an editor, including replacing code, commenting out code, or starting a diff/merge session to either accept or reject changes.

Wing 11 also supports using AI to generate commit messages based on the changes being committed to a revision control system.

You can now also configure multiple AI providers for easier access to different models. However, as of this release, OpenAI is still the only supported AI provider and you will still need a paid OpenAI account and API key. We recommend paying for Tier 2 or better rate limits.

For details see AI Assisted Development under Wing Manual in Wing 11's Help menu.

Package Management with uv

Wing Pro 11 adds support for the uv package manager in the New Project dialog and the Packages tool.

For details see Project Manager > Creating Projects > Creating Python Environments and Package Manager > Package Management with uv under Wing Manual in Wing 11's Help menu.

Improved Python Code Analysis

Wing 11 improves code analysis of literals such as dicts and sets, parametrized type aliases, typing.Self, type variables on the def or class line that declares them, generic classes with [...], and __all__ in *.pyi files.

Updated Localizations

Wing 11 updates the German, French, and Russian localizations, and introduces a new experimental AI-generated Spanish localization. The Spanish localization and the new AI-generated strings in the French and Russian localizations may be accessed with the new User Interface > Include AI Translated Strings preference.

Improved diff/merge

Wing Pro 11 adds floating buttons directly between the editors to make navigating differences and merging easier, allows undoing previously merged changes, and does a better job managing scratch buffers, scroll locking, and sizing of merged ranges.

For details see Difference and Merge under Wing Manual in Wing 11's Help menu.

Other Minor Features and Improvements

Wing 11 also improves the custom key binding assignment user interface, adds a Files > Auto-Save Files When Wing Loses Focus preference, warns immediately when opening a project with an invalid Python Executable configuration, allows clearing recent menus, expands the set of available special environment variables for project configuration, and makes a number of other bug fixes and usability improvements.

Changes and Incompatibilities

Since Wing 11 replaced the AI tool with AI Coder and AI Chat, and AI configuration is completely different than in Wing 10, so you will need to reconfigure your AI integration manually in Wing 11. This is done with Manage AI Providers in the AI menu or the Options menu in either AI tool. After adding the first provider configuration, Wing will set that provider as the default.

If you have questions about any of this, please don't hesitate to contact us at support@wingware.com.

April 01, 2025 01:00 AM UTC

Glyph Lefkowitz

A Bigger Database

A Database File

When I was 10 years old, and going through a fairly difficult time, I was lucky enough to come into the possession of a piece of software called Claris FileMaker Pro™.

FileMaker allowed its users to construct arbitrary databases, and to associate their tables with a customized visual presentation. FileMaker also had a rudimentary scripting language, which would allow users to imbue these databases with behavior.

As a mentally ill pre-teen, lacking a sense of control over anything or anyone in my own life, including myself, I began building a personalized database to catalogue the various objects and people in my immediate vicinity. If one were inclined to be generous, one might assess this behavior and say I was systematically taxonomizing the objects in my life and recording schematized information about them.

As I saw it at the time, if I collected the information, I could always use it later, to answer questions that I might have. If I didn’t collect it, then what if I needed it? Surely I would regret it! Thus I developed a categorical imperative to spend as much of my time as possible collecting and entering data about everything that I could reasonably arrange into a common schema.

Having thus summoned this specter of regret for all lost data-entry opportunities, it was hard to dismiss. We might label it “Claris’s Basilisk”, for obvious reasons.

Therefore, a less-generous (or more clinically-minded) observer might have replaced the word “systematically” with “obsessively” in the assessment above.

I also began writing what scripts were within my marginal programming abilities at the time, just because I could: things like computing the sum of every street number of every person in my address book. Why was this useful? Wrong question: the right question is “was it possible” to which my answer was “yes”.

If I was obliged to collect all the information which I could observe — in case it later became interesting — I was similarly obliged to write and run every program I could. It might, after all, emit some other interesting information.

I was an avid reader of science fiction as well.

I had this vague sense that computers could kind of think. This resulted in a chain of reasoning that went something like this:

human brains are kinda like computers,
the software running in the human brain is very complex,
I could only write simple computer programs, but,
when you really think about it, a “complex” program is just a collection of simpler programs

Therefore: if I just kept collecting data, collecting smaller programs that could solve specific problems, and connecting them all together in one big file, eventually the database as a whole would become self-aware and could solve whatever problem I wanted. I just needed to be patient; to “keep grinding” as the kids would put it today.

I still feel like this is an understandable way to think — if you are a highly depressed and anxious 10-year-old in 1990.

Anyway.

35 Years Later

OpenAI is a company that produces transformer architecture machine learning generative AI models; their current generation was trained on about 10 trillion words, obtained in a variety of different ways from a large variety of different, unrelated sources.

A few days ago, on March 26, 2025 at 8:41 AM Pacific Time, Sam Altman took to “X™, The Everything App™,” and described the trajectory of his career of the last decade at OpenAI as, and I quote, a “grind for a decade trying to help make super-intelligence to cure cancer or whatever” (emphasis mine).

I really, really don’t want to become a full-time AI skeptic, and I am not an expert here, but I feel like I can identify a logically flawed premise when I see one.

This is not a system-design strategy. It is a trauma response.

You can’t cure cancer “or whatever”. If you want to build a computer system that does some thing, you actually need to hire experts in that thing, and have them work to both design and validate that the system is fit for the purpose of that thing.

Aside: But... are they, though?

I am not an oncologist; I do not particularly want to be writing about the specifics here, but, if I am going to make a claim like “you can’t cure cancer this way” I need to back it up.

My first argument — and possibly my strongest — is that cancer is not cured.

QED.

But I guess, to Sam’s credit, there is at least one other company partnering with OpenAI to do things that are specifically related to cancer. However, that company is still in a self-described “initial phase” and it’s not entirely clear that it is going to work out very well.

Almost everything I can find about it online was from a PR push in the middle of last year, so it all reads like a press release. I can’t easily find any independently-verified information.

A lot of AI hype is like this. A promising demo is delivered; claims are made that surely if the technology can solve this small part of the problem now, within 5 years surely it will be able to solve everything else as well!

But even the light-on-content puff-pieces tend to hedge quite a lot. For example, as the Wall Street Journal quoted one of the users initially testing it (emphasis mine):

The most promising use of AI in healthcare right now is automating “mundane” tasks like paperwork and physician note-taking, he said. The tendency for AI models to “hallucinate” and contain bias presents serious risks for using AI to replace doctors. Both Color’s Laraki and OpenAI’s Lightcap are adamant that doctors be involved in any clinical decisions.

I would probably not personally characterize “‘mundane’ tasks like paperwork and … note-taking” as “curing cancer”. Maybe an oncologist could use some code I developed too; even if it helped them, I wouldn’t be stealing valor from them on the curing-cancer part of their job.

Even fully giving it the benefit of the doubt that it works great, and improves patient outcomes significantly, this is medical back-office software. It is not super-intelligence.

It would not even matter if it were “super-intelligence”, whatever that means, because “intelligence” is not how you do medical care or medical research. It’s called “lab work” not “lab think”.

To put a fine point on it: biomedical research fundamentally cannot be done entirely by reading papers or processing existing information. It cannot even be done by testing drugs in computer simulations.

Biological systems are enormously complex, and medical research on new therapies inherently requires careful, repeated empirical testing to validate the correspondence of existing research with reality. Not “an experiment”, but a series of coordinated experiments that all test the same theoretical model. The data (which, in an LLM context, is “training data”) might just be wrong; it may not reflect reality, and the only way to tell is to continuously verify it against reality.

Previous observations can be tainted by methodological errors, by data fraud, and by operational mistakes by practitioners. If there were a way to do verifiable development of new disease therapies without the extremely expensive ladder going from cell cultures to animal models to human trials, we would already be doing it, and “AI” would just be an improvement to efficiency of that process. But there is no way to do that and nothing about the technologies involved in LLMs is going to change that fact.

Knowing Things

The practice of science — indeed any practice of the collection of meaningful information — must be done by intentionally and carefully selecting inclusion criteria, methodically and repeatedly curating our data, building a model that operates according to rules we understand and can verify, and verifying the data itself with repeated tests against nature. We cannot just hoover up whatever information happens to be conveniently available with no human intervention and hope it resolves to a correct model of reality by accident. We need to look where the keys are, not where the light is.

Piling up more and more information in a haphazard and increasingly precarious pile will not allow us to climb to the top of that pile, all the way to heaven, so that we can attack and dethrone God.

Eventually, we’ll just run out of disk space, and then lose the database file when the family gets a new computer anyway.

Acknowledgments

Thank you to my patrons who are supporting my writing on this blog. Special thanks also to Ben Chatterton for a brief pre-publication review; any errors remain my own. If you like what you’ve read here and you’d like to read more of it, or you’d like to support my various open-source endeavors, you can support my work as a sponsor! Special thanks also to Itamar Turner-Trauring and Thomas Grainger for pre-publication feedback on this article; any errors of course remain my own.

April 01, 2025 12:47 AM UTC

March 31, 2025

Ari Lamstein

censusdis v1.4.0 is now on PyPI

I recently contributed a new module to the censusdis package. This resulted in a new version of the package being pushed to PyPI. You can install it like this:

$ pip install censusdis -U

#Verify that the installed version is 1.4.0 
$ pip freeze | grep censusdis 
censusdis==1.4.0

The module I created is called multiyear. It is very similar to the utils module I created for my hometown_analysis project. This notebook demonstrates how to use the module. You can view the PR for the module here.

This PR caused me to grow as a Python programmer. Since many of my readers are looking to improve their technical skills, I thought to write down some of my lessons learned.

Python Files, Modules vs. Packages

The vocabulary around files, modules and packages in Python is confusing. This PR is when the terms finally clicked:

A module is just a normal file with Python code (really). I am not sure why Python invented a new word for this. My best guess is to acknowledge that you can selectively import symbols from a module. This is different than C++ (the first language I programmed in professionally), where #include <file> imports all of the target file.
A package is just a directory that contains Python modules. The best practice appears to be putting a file named __init__.py in the directory to denote that it’s a package. Although the file can be empty and is not strictly necessary (link). This also seems like an odd design decision.

One nice thing about this system is that it allows a package to span multiple (sub)directories. In R, all the code for a package must be in a single directory. I always felt that this limited the complexity of packages in R. It’s nice that Python doesn’t have that limitation.

Dependency Management

Python programmers like to talk about “dependency management hell.” This project gave me my first taste of that.

The initial version of the multiyear module used plotly to make the output of graph_multiyear interactive. I used it to do exploratory data analysis in Jupyter notebooks. However, when I tried to share those notebooks via github the images didn’t render: apparently Jupyter notebooks in github cannot render Javascript. The solution I stumbled upon is described here and requires the kaleido package.

The issue? Apparently this solution works with kaleido v0.2.0, but not the latest version of kaleido (link). So anyone who wants this functionality will need to install a specific version of kaleido. In Python this is known as “pinning” a dependency.

Technically, I believe you can do this by modifying the project’s pyproject.toml file by hand. But in practice people use tools like uv or poetry to both manage this file and create a “lockfile” which states the exact version of all packages you’re using. In this project I got experience doing this with both uv (which I used for my hometown_analysis repo) and poetry (which censusdis uses).

Linting

At my last job I advocated for having all the data scientists use a Style Guide. At that company we used R, and people were ok giving up some issues of personal taste in order to make collaboration easier. The process of enforcing adherence to a style guide (or running automated checks on code to detect errors) is called “linting”, and it’s a step we did not take.

In my hometown_analysis repo I regularly used black for this. It appears that black is the most widely used code formatter in the Python world. It was my first time using it on a project, and I simply ran it myself prior to checking in code.

The Censusdis repo takes this a step further:

In addition to running black, it recommends contributors also run flake8 and ruff on their code prior to making a PR. For better or for worse, Python seems to have a lot of tools that do linting. There appears to be some overlap in what they all do, and I can’t speak to the unique differences between them. One thing that surprised me is that at least one of them was particular about the format in which I wrote documentation for my functions.
It automatically runs all of these tools on each PR using Github Actions (link). If any of the linter tools detects an issue it causes the PR to fail the automated test suite.

Automated Tests

Speaking of tests: I did not feel the need to write them for my utils module for the hometown_analysis project. But censusdis uses pytest and has 99% test coverage (link). So it seemed appropriate to add tests to the multiyear module.

Writing tests is something that I’ve done occasionally throughout my career. Pytest was covered in Matt Harrison’s Professional Python course that I took last year, but I found that I forgot a lot of the material. So I did what most engineers would do: I looked at examples in the codebase and used an LLM to help me.

Type Annotations

I have mixed feelings about Python’s use of Type Annotations.

I began my software engineering career using C++, which is a statically typed language. Every variable in a C++ program must have a type defined at compile time (i.e. before the program executes). Python does not have this requirement, which I initially found freeing. Type annotations, I find, remove a lot of this freedom and also make the code a bit harder to read.

That being said, the censusdis package uses them throughout the codebase, so I added them to my module.

In Professional Python I was taught to run mypy to type check my type annotations. While I believe that my code passed without error, I noticed that the project had a few errors that were not covered in my course. For example:

cli/cli.py:9: error: Skipping analyzing "geopandas": module is installed, but missing library stubs or py.typed marker

It appears that type annotations become more complex when your code uses types defined by third-party libraries (such as Pandas and, in this case, GeoPandas). I researched these errors briefly and created a github issue for them.

Code Review

A major source of learning comes when someone more experienced than you reviews your code. This was one of the main reasons I chose to do this project: Darren (the maintainer of censusdis) is much more experienced than me at building Python packages, and I was interested in his feedback on my module.

Interestingly, his initial feedback was that it would be better if the graph_multiyear function used matplotlib instead of plotly. Not because matplotlib is better than plotly, but because other parts of censusdis already use matplotlib. And there’s value in a package having consistency in terms of which visualization package it uses. This made sense to me, although I do miss the interactive plots that plotly provided!

Conclusion

The book Software Engineering at Google defines software engineering as “programming integrated over time.” The idea is that when code is written for a small project, software engineering best practices aren’t that important. But when code is used over a long period of time, they become essential. This idea stayed with me throughout this project.

- The first time I did a multi-year analysis of ACS data was for my Covid Demographics Explorer, which I completed last June. I considered the project a one-off. I wrote a single script to download the data and an app to visualize it.
- For my hometown_analysis project I wanted to do exploratory data analysis of several variables over time. So I wrote a handful of functions to download and visualize multi-year ACS data. I put all the code in a single module and pinned the dependencies. I wrote docstrings for all the functions. I reasoned that if I ever want to do a similar analysis in the future then I could reuse the code.
- When I wanted to make it easier for others to use the code I added it to an existing package. That required being more rigorous about coding style, adding automated tests and type annotations. It also required me to make design decisions that are best for the overall package, even when they conflict with design decisions I made when working on the module independently.

My impression is that a lot of Python programmers (especially data scientists) have never contributed their code to an existing package. If you are given the opportunity, then I recommend giving it a shot. I found that it helped me grow as a Python programmer.

While I have disabled comments on my blog, I welcome hearing from readers. Use this form to contact me.

March 31, 2025 04:00 PM UTC

Real Python

Python's Bytearray: A Mutable Sequence of Bytes

Python’s bytearray is a mutable sequence of bytes that allows you to manipulate binary data efficiently. Unlike immutable bytes, bytearray can be modified in place, making it suitable for tasks requiring frequent updates to byte sequences.

You can create a bytearray using the bytearray() constructor with various arguments or from a string of hexadecimal digits using .fromhex(). This tutorial explores creating, modifying, and using bytearray objects in Python.

By the end of this tutorial, you’ll understand that:

A bytearray in Python is a mutable sequence of bytes that allows in-place modifications, unlike the immutable bytes.
You create a bytearray by using the bytearray() constructor with a non-negative integer, iterable of integers, bytes-like object, or a string with specified encoding.
You can modify a bytearray in Python by appending, slicing, or changing individual bytes, thanks to its mutable nature.
Common uses for bytearray include processing large binary files, working with network protocols, and tasks needing frequent updates to byte sequences.

You’ll dive deeper into each aspect of bytearray, exploring its creation, manipulation, and practical applications in Python programming.

Get Your Code: Click here to download the free sample code that you’ll use to learn about Python’s bytearray data type.

Take the Quiz: Test your knowledge with our interactive “Python's Bytearray” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Python's Bytearray

In this quiz, you'll test your understanding of Python's bytearray data type. By working through this quiz, you'll revisit the key concepts and uses of bytearray in Python.

Understanding Python’s `bytearray` Type

Although Python remains a high-level programming language, it exposes a few specialized data types that let you manipulate binary data directly should you ever need to. These data types can be useful for tasks such as processing custom binary file formats, or working with low-level network protocols requiring precise control over the data.

The three closely related binary sequence types built into the language are:

bytes
bytearray
memoryview

While they’re all Python sequences optimized for performance when dealing with binary data, they each have slightly different strengths and use cases.

Note: You’ll take a deep dive into Python’s bytearray in this tutorial. But, if you’d like to learn more about the companion bytes data type, then check out Bytes Objects: Handling Binary Data in Python, which also covers binary data fundamentals.

As both names suggest, bytes and bytearray are sequences of individual byte values, letting you process binary data at the byte level. For example, you may use them to work with plain text data, which typically represents characters as unique byte values, depending on the given character encoding.

Python natively interprets bytes as 8-bit unsigned integers, each representing one of 256 possible values (2⁸) between 0 and 255. But sometimes, you may need to interpret the same bit pattern as a signed integer, for example, when handling digital audio samples that encode a sound wave’s amplitude levels. See the section on signedness in the Python bytes tutorial for more details.

The choice between bytes and bytearray boils down to whether you want read-only access to the underlying bytes or not. Instances of the bytes data type are immutable, meaning each one has a fixed value that you can’t change once the object is created. In contrast, bytearray objects are mutable sequences, allowing you to modify their contents after creation.

While it may seem counterintuitive at first—since many newcomers to Python expect objects to be directly modifiable—immutable objects have several benefits over their mutable counterparts. That’s why types like strings, tuples, and others require reassignment in Python.

The advantages of immutable data types include better memory efficiency due to the ability to cache or reuse objects without unnecessary copying. In Python, immutable objects are inherently hashable, so they can become dictionary keys or set elements. Additionally, relying on immutable objects gives you extra security, data integrity, and thread safety.

That said, if you need a binary sequence that allows for modification, then bytearray is the way to go. Use it when you frequently perform in-place byte operations that involve changing the contents of the sequence, such as appending, inserting, extending, or modifying individual bytes. A scenario where bytearray can be particularly useful includes processing large binary files in chunks or incrementally reading messages from a network buffer.

The third binary sequence type in Python mentioned earlier, memoryview, provides a zero-overhead view into the memory of certain objects. Unlike bytes and bytearray, whose mutability status is fixed, a memoryview can be either mutable or immutable depending on the target object it references. Just like bytes and bytearray, a memoryview may represent a series of single bytes, but at the same time, it can represent a sequence of multi-byte words.

Now that you have a basic understanding of Python’s binary sequence types and where bytearray fits into them, you can explore ways to create and work with bytearray objects in Python.

Creating `bytearray` Objects in Python

Unlike the immutable bytes data type, whose literal form resembles a string literal prefixed with the letter b—for example, b"GIF89a"—the mutable bytearray has no literal syntax in Python. This distinction is important despite many similarities between both byte-oriented sequences, which you’ll discover in the next section.

The primary way to create new bytearray instances is by explicitly calling the type’s class constructor, sometimes informally known as the bytearray() built-in function. Alternatively, you can create a bytearray from a string of hexadecimal digits. You’ll learn about both methods next.

The `bytearray()` Constructor

Read the full article at https://realpython.com/python-bytearray/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

March 31, 2025 02:00 PM UTC

Planet Python

April 05, 2025

Embeddings

The word2vec CBOW architecture

JAX implementation

Training

Extracting embeddings and finding word similarities

Optimizations

Original word2vec code

Modern text embeddings

Full code

Live streams

English

Spanish / Español

Portuguese / Português

Weekly office hours

April 04, 2025

Enhanced Python development using Copilot and Notebooks

Improved support for editable installs

Faster and more reliable diagnostic experience (Experimental)

Pylance custom Node.js arguments

We would also like to extend special thanks to this month’s contributors:

April 03, 2025

Preface: the migration to pytest

pytest invocation is slow

Where is time being spent?

Disable plugin auto loading

Handling pytest-xdist

Disable some default plugins

Optimizing test collection time

Putting it all together

Other links which may be useful

No More “Going Forward” Caveats

Microservices Without the Complexity

Blazing fast views

It Just Feels Right

Setting Up the Working Environment

Building the Notes GUI

Styling our notes

Remove Window Decorations

Movable notes

Multiple notes

Adding Notes to the Tray

Adding a Menu

Setting up the Notes database

Integrating the Data Model into our UI

Starting up

Conclusion

New physical cartridges that don't work offline

Switch 2 might be expensive?

Switch 2 requires microSD Express cards

"Single-Pak" / "Download Play" is back with GameShare

Switch 2 will be the console of GameCube?

April 02, 2025

How to Use Python’s .strip() Method to Remove Whitespace From Strings

Remove Specific Characters With .strip()

CVE-2025-27556: Potential denial-of-service vulnerability in LoginView, LogoutView, and set_language() on Windows

Affected supported versions

Resolution

CVE-2025-27556: Potential denial-of-service vulnerability in LoginView, LogoutView, and set_language() on Windows

The following releases have been issued

General notes regarding security reporting

Installing Streamlit

Add title, headings, and paragraphs

Adding Titles

Adding Headings

Adding Paragraphs

Adding different kinds of buttons to the Streamlit app

Basic Button

Download Button

Radio Buttons for Options

Adding slider

Adding the dropdown menu in the app

Adding a Sidebar in Streamlit

Creating a simple web app using Streamlit

April 01, 2025

Articles & Tutorials

Projects & Code

Events

Creating a Vertical Layout

How to Use Python’s `.strip()` Method to Remove Whitespace From Strings

Remove Specific Characters With `.strip()`

CVE-2025-27556: Potential denial-of-service vulnerability in `LoginView`, `LogoutView`, and `set_language()` on Windows

CVE-2025-27556: Potential denial-of-service vulnerability in `LoginView`, `LogoutView`, and `set_language()` on Windows