I’ve recently had a nerdy discussion with a colleague during a code review about when to use Pythons pass statement vs. the ellipsis literal .... You’ve probably seen the pass statement in Python code as a means to indicate that a block is intentionally left empty to avoid a SyntaxError. Like in the following except block where we expect that an exception might be raised but just want to ignore it:

try:
    client.get(id=42)
except ApiException:
    pass

Somehow the ellipsis literal ... has gotten some hype lately and some people started doing this:

try:
    client.get(id=42)
except ApiException:
    ...

While this is syntactically valid Python code the semantics are weird to me. To me, the pass statement effectively communicates that we should just pass this block without running any code at all. While the ... are more like "there is something to be expected here in the future”. Therefore, the ... can be thought of a placeholder. Whenever I see the ... my head auto-plays a Dun Dun Dun sound effect - to emphasize the suspension of code. It would be a pity if it’s a suspension of code until all eternity if you use at as a pass replacement.

Where should I use the ...?

As I’ve already mentioned you should use the ... as a placeholder where eventually some could should appear, but you don’t necessarily want to raise NotImplementedError. A similar situation where I use it is with Pythons Abstract Base Classes to indicate that the function body needs to be implemented by a subclass.

from abc import abstractmethod

class AbstractEventLoop:
    @abstractmethod
    def run(self, ...):
        ...

The next place where I use it is in Python stub files to indicate an ignored function body:

def get(id: int) -> Model: ...

The fourth place is in extended slicing with custom container types, for what the ... was originally introduced for. For example, the numpy array indexing supports it:

from numpy import arange

a = arange(16).reshape(2, 2, 2, 2)
flat_a = a[..., 0].flatten()

But what is the ... literal ?

The ellipsis literal ... is syntactic sugar for the Ellipsis object, which is the sole instance of the types.EllipsisType type. You can use it like every other object in Python, except that you can’t create a new one. Thus, when you override it (yes, that’s possible: __builtins__.Ellipsis = 42) you can’t really get it back - thus, don’t do it.

I started, what turns out to be, a long running journey to migrate away from Google. Part of that journey is my personal email address, calendar and address book.

After a lengthy back and forth and changing the provider multiple times, I finally (hopefully?!) settled with Migadu.com. I will probably write another post about why Migadu and how to actually go about such a migration but this one is just about how to properly setup auto-discovery for their calendar and address book offerings.

Migadu doesn't really advertise that they even support calendar and contacts. The only thing you'll find on their website is this:

We make the basic CalDAV and CarDAV services available, but they are not our focus. We continue developing them but please do not expect we will ever compete with dedicated calendar services.

They are using sabre/dav and make it available at cdav.migadu.com. Since for Migadu you always have to bring your own domain, you can configure two SRV entries in your DNS settings for your domain. This will allow calendar and address book tooling to auto-discover your CalDav and CardDav settings given your email address.

RFC6765 defines how to do that. There are two ways: a /.well-known location and the aforementioned SRV entry. You already have to configure your DNS for the email services anyways, so lets use that.

The following two SRV entries are required:

_caldavs._tcp.example.com.  3000 IN TXT "path=/calendars"
_carddavs._tcp.example.com. 3000 IN TXT "path=/calendars"
_caldavs._tcp.example.com.  3000 IN SRV 0 1 443 cdav.migadu.com
_carddavs._tcp.example.com. 3000 IN SRV 0 1 443 cdav.migadu.com

Replace the example.com domain with your own domain. The two additional TXT entries are required, because the CalDav / CardDav server is hosted at the /calendars path.

In Go it's very common to see the pattern that after creating a sync.WaitGroup() the Wait() call is immediately defer-red, like this:

func f() {
  var wg sync.WaitGroup
  defer wg.Wait()

  // rest of the logic
}

We need to be careful with this pattern due to how and when Go captures return values. For example, consider the following function that does some concurrent processing and using a sync.WaitGroup before returning the processed data to the caller:

func f(xs []int) map[int]string {
  var mu sync.Mutex
  var wg sync.WaitGroup
  defer wg.Wait()

  results := make(map[int]string, len(xs))

  for _, x := range xs {
    wg.Add(1)
    go func() {
      defer wg.Done()
      mu.Lock()
      defer mu.Unlock

      // some calculations
      results[x] = calc(x)
    }()
  }

  return results
}

This works fine, but consider the same using a string slice as return value, like this:

func f(xs []int) []string {
  var mu sync.Mutex
  var wg sync.WaitGroup
  defer wg.Wait()

  results := make([]string, 0, len(xs))

  for _, x := range xs {
    wg.Add(1)
    go func() {
      defer wg.Done()
      mu.Lock()
      defer mu.Unlock

      // some calculations
      results = append(results, calc(x))
    }()
  }

  return results
}

This will lead to a race condition. The problem is that the function f will (more or less) return immediately and capture the pointer to results, while the goroutines make an assignment to results.

You can avoid this problem in multiple ways:

1. do not defer the Wait() call, but call it manually right before the return.
2. do not assign to the return variable in the goroutines. Instead use the index assignment, like results[i] = calc(x).
3. use a named return value.

(1): manual calling Wait

Manually deferring the Wait() is very easy, to code example above would change to this:

func f(xs []int) []string {
  var mu sync.Mutex
  var wg sync.WaitGroup

  results := make([]string, 0, len(xs))

  for _, x := range xs {
    wg.Add(1)
    go func() {
      defer wg.Done()
      mu.Lock()
      defer mu.Unlock

      // some calculations
      results = append(results, calc(x))
    }()
  }

  wg.Wait()

  return results
}

(2): do not assign to the return variable

Since the underlying problem illustrated here is about the assignment to the captured return variable, we may just not re-assign it in goroutines:

func f(xs []int) []string {
  var mu sync.Mutex
  var wg sync.WaitGroup
  defer wg.Wait()

  results := make([]string, 0, len(xs))

  for i, x := range xs {
    wg.Add(1)
    go func() {
      defer wg.Done()
      mu.Lock()
      defer mu.Unlock

      // some calculations
      results[i] = calc(x)
    }()
  }

  return results
}

In this example, this may work because we know the length of the returned slice ahead of time, but that may not always be possible.

(3): use a named return

We can also use a named return which changes when Go captures the return value. The following works just fine:

func f(xs []int) (results []string) {
  var mu sync.Mutex
  var wg sync.WaitGroup
  defer wg.Wait()

  for _, x := range xs {
    wg.Add(1)
    go func() {
      defer wg.Done()
      mu.Lock()
      defer mu.Unlock

      // some calculations
      results = append(results, calc(x))
    }()
  }

  return results
}

Conclusion

Pay attention when using sync.WaitGroup and deferring Wait() calls. Make sure your return values or whatever is awaited with the wait group doesn't depend on simple variable assignments.

I've already hinted in Migadu.com CalDav and CardDav auto-discovery that I've moved my personal email, calendar and contacts away from Google to Migadu.

A little more than a month has passed now and I want to briefly recap of what went well and what didn't.

E-Mail

Migadu's primary focus is email and so far I've been very pleased with the on-boarding process, like the DNS setup for my domains and the admin interface overall.

Migrate emails

The migration of my Gmail address was really straight-forward by using Imapsync. However, since that particular address has been in used for 15 years it accumulated quite a bunch of emails and Gmails All Mail folder that contains a copy of each and every email in all folders (yes, their labels are folders with copies of your emails) didn't particularly help with the mailbox size. I took the better half of a week to fully sync my mailbox from Gmail to Migadu. I'd also recommend to use imapsync --dry ... to verify what imapsync will perform.

Support for plus addressing

Support for plus addressing was a must for me, but the default configuration in Migadu is a little weird to me taste. They basically auto-create folders for the detail part. So emails sent to timo+test@furrer.life would automatically end up in an IMAP folder called test. That folder is also auto-created, meaning that strangers are allowed to create IMAP folders by simply sending an email. I'm not sure under what circumstances someone would ever want that. Anyways, they document how to disable it. Make sure though that you account for the rewrites in your Sieve scripts, because the Delivered-To header will then point to timo@furrer.life instead of timo+test@furrer.life. Luckily, we have the X-Envelope-To header to use (that contains the SMTP RCT header value) - although you cannot use the envelope extension, because of their architecture and how they delivery emails to your actual mailbox via proxies.

Support for Sieve

Sieve is a language to filter emails based on conditional logic - basically a bunch of ifs. In case you have ever used Sieve, you wouldn't want to trade it with e.g. Gmail's filters. Migadu hosts a managed sieve endpoint that you can enable access for on an individual mailbox basis. You may use any sieve client to connect to it with your mailbox credentials. I'd recommend sieve-connect to upload the Sieves scripts. It's probably worth pointing out that you should test your scripts in a separate mailbox to not loose any incoming emails by accident.

You can easily upload and activate a sieve script without using the REPL, like this:

sieve-connect -s imap.migadu.com -u timo@furrer.life --localsieve ./main.sieve --upload
sieve-connect -s imap.migadu.com -u timo@furrer.life --localsieve ./main.sieve --activate

As mentioned in Support for plus addressing above, you can't use the envelope Sieve extension. According to the Migadu support that's because dovecot and postfix are not on the same servers. I'm not expert enough to actually tell if it wouldn't somehow be possible to still use or support the envelope extension, but yeah, it's a bummer.

For example, if envelope would be supported you could easily refile to a folder like this:

if envelope :detail "to" "test" {
  fileinto "test";
}

Instead we have to resort to other headers, like X-Envelope-To:

if header :contains "X-Envelope-To" "timo+test@furrer.life" {
  fileinto "test";
}

Calendars and Contacts

Migadu officially states that they only have very basic CalDav and CardDav support:

We make the basic CalDAV and CarDAV services available, but they are not our focus. We continue developing them but please do not expect we will ever compete with dedicated calendar services.

And after using it for a while I can confirm exactly that. The problem is calendar scheduling, that is, replying to events and receiving replies from attendees on events that I'm organizing. Migadu right now supports neither. They are specifically missing support for iMIP and iTIP. So far, that's been the most limiting factor of the migration. Right now, I'm manually updating attendee replies and sending out replies for my own participation status.

Conclusion

I still have plenty of things to explore and address. Especially the shortcomings in calendar scheduling support. I'm exploring other hosting options for CalDav and CardDav, preferably self-hosted. However, I'm also tempted to implement a little bridging services for iMIP and iTIP.

A couple of weeks ago, Chris Morgan published I've banned query strings. I read it, liked it and then did roughly the same thing on my own site - with two deliberate differences.

Chris's opening sums up the motivation better than I could:

I don't like people adding tracking stuff to URLs. Still less do I like people adding tracking stuff to my URLs.

[...] UTM parameters are for me to use, not you. Leave my URLs alone.

The premise is the same here. A ?utm_source=... or ?ref=... tacked onto one of my URLs by some intermediary is, at best, noise I never asked for and at worst a tracker the referrer is using to nudge my visitor's behaviour into a funnel. I'd rather refuse to serve those requests than pretend they're a legitimate way to reach a page on my site. I'm also a fan of having one true canonical URL to all my pages.

Where I differ from Chris

1. cache-busters like ?v=<digits> are allowed

Chris went for a true blanket ban - including breaking old cache-busting URLs like ?t=... and ?h=... that his site used to serve. That's likely the right call when none of those URLs are still in circulation. In any case, those might only be used for static assets anyways, where people don't have bookmarks to.

My situation is slightly different: I actively use ?v=<n> as a cache buster on assets I serve today. The very HTML you're reading links to main.css?v=1. I use it so that I can set a very high Cache-Control: max-age: ... on static assets. If I matched Chris's strictness I'd have to either give up on query-string cache busting (and switch to fingerprinted filenames or Cache-Control juggling), or break my own page load on every bump.

So my rule is a narrow allowlist: everything is blocked, except ?v=<digits>. The matcher is intentionally strict - the whole query string must be exactly v= followed by digits, nothing else, no extra parameters smuggled in alongside.

(no_query_strings) {
    @bad_query `{http.request.orig_uri}.contains("?") && !{http.request.uri.query}.matches("^v=[0-9]+$")`
    error @bad_query 403
}

The first clause uses orig_uri so a bare trailing ? still trips the ban - Caddy's {query} placeholder can't distinguish "absent" from "empty", and a lone ? deserves the same treatment as a parameter list. The second clause uses the canonical {uri.query} because Caddy doesn't expose .query as a sub-key on orig_uri - the rewrite never touches the query so the two are equivalent here.

2. 403 Forbidden, not 414 URI Too Long

Chris picked 414 URI Too Long, and is upfront about it:

You could argue that I'm abusing 414 URI Too Long. I respond that it's funnier this way.

It's indeed nice, but I wanted to pick a status code that I can defend on RFC grounds rather than vibes. Here's how I read RFC 9110 and RFC 7725 for this case:

400 Bad Request

The server cannot or will not process the request due to something that is perceived to be a client error (e.g., malformed request syntax). The request isn't malformed; ?utm_source=x is perfectly well-formed. Too generic.

403 Forbidden

The server understood the request but refuses to authorize it. That is exactly what's happening: I understood the request, the URL would otherwise resolve, and I am refusing on policy grounds. The spec also explicitly notes that a server can describe that reason in the response content, which is what the body of the 403 page does.

404 Not Found

Misleading. The resource exists; I just won't serve it via this URL. Also has unpleasant SEO and caching side effects.

414 URI Too Long

A refusal to service the request because the request-target is longer than the server is willing to interpret. The objection is about length, not policy or content. Although, I agree that one could argue that everything after the canonical URL is too long.

451 Unavailable For Legal Reasons

Not legal reasons. Just personal taste.

403 is the cleanest semantic match. I'm not 100% certain, but I'd interpret the "authorize" from RFC9110 as not only HTTP authentication or authorization, but rather the more general sense of "permit".

(Okay, Chris is right that 414 is funnier, though. I'll concede that one.)

What it looks like

When a request comes in with anything other than ?v=<digits>, Caddy short-circuits with a 403 and serves a small explainer page. You can try it yourself: furrer.life/~timo/?utm_source=this-post. The page tells you what happened, why, and offers the same URL without the query string.

If you want to follow Chris down this path, his post links the relevant Caddyfile snippet on his site or use mine above which is a small variant of that with the extra allowlist clause shown above.