Define a stability policy

[JelleZijlstra]

We're often hesitant to change formatting now, even though technically we're still in beta (#517). Some questions we should document the answer to:

• When does Black accept formatting changes? How is the decision to change formatting made?
• What are the guarantees Black wants to continue making? When will we make exceptions to the AST safety guarantee? (XREF: Latest black changes ASTs #2150)
• As a user, what should I expect when upgrading to a new version of Black?
• How will this change when we have a "stable" release? Will anything change?

The answers to these questions should be in the documentation. Feel free to respond to this issue with other similar questions.

[graingert]

Black is stable once Django adopts it

[carltongibson]

Black is stable once Django adopts it

😀

Just to comment on that, it was a point at the time Django agreed to adopt Black that we would wait for the stable release. That we haven't adopted it is because no such release has come.

So from our perspective it would be good to reach that point.

It's been a couple of years now, so I suspect resistance to adopting Black (at all) has probably shrunk considerably, but from the outside it's not easy to see what's left to do for a stable version.

I think it has to be acceptable to leave open the possibility of future changes, so stable can't just mean frozen.

If there were a clear statement, it might allow us to decide to adopt Black anyway, even though a stable version wasn't yet released.

Of course that conversation might not go well, but at the moment it's hard to even find grounds to open it.

I hope that's of some use.

Thanks for the great efforts, and the great project! 🏅

[akhmerov]

Linking #1256 as related.

[pradyunsg]

I don't have an opinion on the first two as a user -- those are decision making process items and are a maintenance question so IMO purely for the maintainers to decide. :)

For the other two, a proposal I have is to promise something similar to isort: https://pycqa.github.io/isort/docs/major_releases/release_policy.html#formatting-guarantees

For Black, IMO that sould look something along the lines of:

Black guarantees that formatting will stay the same for the options given, in accordance to its test suite, over the course of a calendar year. This means projects can safely use black ~= 21.0 without worrying about major formatting changes disrupting their project in 2021.

It would also be useful to mention earlier on the page that:

Black follow Calendar Versioning -- the version number reflects when a release has been made.

[pradyunsg]

This gives the users a year of peaceful use with "will the format change?" only being a concern at the start of every year -- there are no promises that things will change but also none that things won't. This is a significantly more dilute promise than "it will never change, except bugs" but that also means you can make improvements and don't have to argue about "is that a bug" with users. I think y'all are very aware that changes in black's behaviours are very disruptive, so I'm willing to trust that you won't make horrible choices just because you have this option.

With such a policy, it would also make sense to have a --i-want-next-year-stuff-now flag (with a better name please) to facilitate early adoption -- which will help weed out bugs as well as be a nice mechanism for users to try out the upcoming changes.

Of course, this ties improvements to the calendar year, but honestly, I kinda like that. Plus, with Python itself moving to an annual cadence and black becoming more and more stable, I feel like that's fine. Plus, I can't think of any other approach to use version numbers to communicate compatibility; unless you're open to changing the versioning scheme. :)

[RonnyPfannschmidt]

Given that all modern usages of black I am aware of use pinning,

Even if there are necessary changes, people would have those as part of the pin update in a controlled manner.

So i really don't understand why anyone still has a problem with black changing on occasion.

It's rare and well manageable.

[carltongibson]

So, could this be closed by just stating that on the Readme/docs and removing the beta modifier from the version number? (Is Black really still Beta? It doesn't seem that way but... 🤔)

[JelleZijlstra]

@carltongibson there is a separate issue about having a non-beta release. But removing the b from the release number (which I agree isn't very meaningful) won't magically answer the questions I posed above.

[carltongibson]

@JelleZijlstra Sure. I was probably being overly rhetorical. Sorry.

I think is a good topic to be resolved (i.e. what counts as stable) — it's hard to see from the outside, and I know we're just one project, but we've got stuck in a position where, having agreed to wait for the stable release, we're not currently able to adopt. (That's not your issue of course, but I'm hoping a resolution here will allow us to push forwards.)

I'll leave it now. I wanted to add a big 👍 with some context, rather than distract. Thanks again.

[pradyunsg]

I think what @carltongibson and @RonnyPfannschmidt are saying won't answer the first two questions, but they do definitely answer the other two, as I read it.

• As a user, what should I expect when upgrading to a new version of Black?

Whenever I want to, and feel comfortable doing so.

• How will this change when we have a "stable" release? Will anything change?

Not at all. The guarantees are same as what they are today.

---

Here's my try at phrasing what they're suggesting as a compatibility policy:

Each release of Black MAY format your code differently and users should be mindful of that, when updating to a new version of black. While these changes are minimised to avoid disruption, users are expected to pin the Black version they use if they believe they are sensitive to potential format changes.

---

I think this is definitely the implicit contract right now. It is probably also diametrically opposite to what some folks might want -- but I guess the point is that there's no need to be changing anything beyond documenting this implicit contract.

[felix-hilden]

Thank you for all the discussion so far. I started wondering, given our choice of calendar versioning, what would be the best way of communicating the degree of disruption. Semantic versions (or if not strictly semantic, simply ordinary version numbers) are more obvious in that regard, but we won't have such luxury. Would it suffice to describe the changes in release notes and expect users to thoroughly review them? I mean, that's a perfectly valid option! And maybe the only good one. A short summary of changes or their scope might also help users.

[JelleZijlstra]

I'm warming up to @pradyunsg's idea above, which I'd detail as follows: We provide a stability guarantee for one year, plus an --unstable option that provides new formatting changes that we like. At the beginning of each new year, we provide a release that turns the unstable behavior into the default. --unstable will then do nothing until we come up with something else we want to change. We guarantee that (as long as you don't use --unstable), Black will format any valid Python code the same way for all releases in the same year, so you can safely upgrade (and get any stability fixes/new options/integration fixes you want).

[felix-hilden]

That'd be quite good in my opinion too! And having the option, rather than some separate unstable releases, should also be smoother for development. Concretely then, I imagine our release notes would have an "Unstable" section, and new year's releases could pull them all to one big changelog. This all seems very reasonable, since it won't hinder development behind the unstable flag at all! Hooray for SemCalVer 😄

When does Black accept formatting changes? How is the decision to change formatting made?

I think it should be whenever the new style is decidedly better, by whatever metric we choose. Otherwise we would be cornering ourselves once more. "How?", is the harder question. I think the current approach of communication, consensus and Łukasz' veto have been working well.

What are the guarantees Black wants to continue making? When will we make exceptions to the AST safety guarantee?

In my opinion the current exceptions are fine. Some programs might rely on strict docstring formatting and such, but the gain does outweigh the potential problems. However, as @pradyunsg suggested on Discord with changing the indent of textwrap.dedent with multiline strings, more of these AST-unsafe-but-probably-safe-overall transforms exist. If we were to start implementing them, I'd really like to see a flag disabling all AST-unsafe transformations (some discussion) or enabling them, either way. But, I also feel like it could be a slippery slope, although I know additional transformations like import reordering are a highly-requested feature (#333). Personally I like clear separation of concerns, because there are tools like Tox and pre-commit that can be used to combine functionality across many tools.

How will this change when we have a "stable" release? Will anything change?

With all of this in mind, I don't see a clear reason to keep Black in beta. So there could be two options of performing our first non-beta release. Right away: let's communicate it's nothing special and document our stability and version support policies, OR next january, when we would have more time to set things up and prepare users.

It would be really nice to hear what @ambv thinks of all this!

[graingert]

I imagine our release notes would have an "Unstable" section, and new year's releases could pull them all to one big changelog.

I'd like to see some level of stability for the unstable flag, eg if black were to hesitate at the end of the year or roll back formatting decisions it would still break trust in it.

Perhaps --preview would be a better name?

[graingert]

also new syntax changes in the input source code, like parenthesized with or match, can automatically opt into new formatting styles and features because black makes no promises where it previously produced no output

[felix-hilden]

I'd like to see some level of stability for the unstable flag [..] Perhaps --preview would be a better name?

Oh, do you mean only in the naming? Sure, preview sounds more exciting! And I do agree that the flag shouldn't be abused for testing all kinds of different formatting styles until we decide on what's best. I'd see it as "we're committing to this somewhat disruptive style because it's better than the one we had".

[srittau]

As a user, a semver-like versioning seems the best understood: Guarantee format stability during a major version (for a set of options). It's possible to add forward compatibility by using options. Minor versions can add features, like support for new syntax constructs (think match statement) or options. Micro versions are bugfixes. This would allow users to specify version ranges as e.g. black >=2.3, black <3, similar to libraries. I don't think using calver is a good match.

[srittau]

I don't think using calver is a good match.

To expand on that. This would mean that minor or urgent fixes are not possible at the start of a year. Basically, the black project would need to have a big push to get changes in at that point or block all further releases. It basically forces contributors to finish features before the start of the new year, traditionally a very hectic time, to have a chance to get a feature in.

For users, using calver to signal "API" stability is quite unusual. Calver is usually on projects, where stability guarantees are not necessary, for example apps or libraries with a rock-solid API that mainly updates data.

[ambv]

Rust does it though, no? This would be like Rust "editions".

[gsemet]

Hi. CalVer is legit, 0ver is legit, semver is legit. These three formats should cover basically every people need:

• a project is mature enough (or not-so complex) that it is willing and able to guarantee the semantic-versioning contract -> enjoy using semver
• a projet does not want (or it is simply too costly to do) to deal with compatibility: calver or zero-ver fits.

I can see black follow straight forward Calver YY.0M.PATCH. Guess what, it is already doing it, simply hiding with all this stability/governance discussions. Just split the issues into smaller ones, pick one versioning scheme, publish non-prerelease packages and THEN implement all policies you want moving forward.

My libs uses semver and works great. But most of the CLI tools I provides use 0-ver or calver, both are great, because guaranteeing backward compatibility when the tool is complex can be hard (and costly) to achieve. Effort can be put elsewhere.

[felix-hilden]

Just split the issues into smaller ones, pick one versioning scheme, publish non-prerelease packages and THEN implement all policies you want moving forward.

Please don't take the maintenance of this library as granted or suggest things as if they are obvious. We try to do our best here and make smart decisions. It's not like we try to antagonise users and try to prolong releases just for the sake of it.

Do you have any suggestions to change the policy that was discussed above?

[gsemet]

Publish next version as 21.12.0 ?

[pauloxnet]

Black is stable once Django adopts it

So now Black is really stable since Django adopted black with this PR
django/django#15387