Establish the principles of a service-oriented architecture - #862
mike-gorman-bitwarden wants to merge 26 commits into
Conversation
Codifies the ten decisions that define what a service is and how it behaves under a service-oriented architecture: where boundaries come from, who owns a resource's data store, how a boundary is crossed, when a local copy is permitted, and what every service publishes. Recorded as Proposed so the decisions can be discussed and refined before ratification. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🤖 Bitwarden Claude Code ReviewOverall Assessment: APPROVE At head this PR adds exactly one file, Code Review DetailsNo new findings. Considered and dropped:
|
Deploying contributing-docs with
|
| Latest commit: |
ecb9f3c
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://6ca3e477.contributing-docs.pages.dev |
| Branch Preview URL: | https://docs-adr-0035-service-orient.contributing-docs.pages.dev |
Clarified the breaking changes policy and API versioning process for services.
theMickster
left a comment
There was a problem hiding this comment.
I think we have a good start and are 80% of the way there to a great ADR.
I have noted some specific concerns with textbook AI writing fingerprints that should be addressed; but let's be sure to re-read the whole document to pull out anything else.
We have a lot of the [X] rather than [Y] writing style that we should review and consider if each one is required. I realize that these statements are important parts of an ADR because we are comparing decision options, but there's like 10+ uses of the writing style and they cannot all be required for the ADR to stand strong.
I also think we can cut many of the overly verbose and run-on statements throughout the ADR without loosing technical merit.
Co-authored-by: Mick Letofsky <mletofsky@bitwarden.com>
…r to service-oriented-architecture
…rchitecture' into docs/adr-0035-service-oriented-architecture
There was a problem hiding this comment.
I pulled down the branch and re-read it after running the project. I think we have a few grammatical improvements below to make.
One point of confusion that I am unsure whether it matters to solve in this document is that we reference 'rule # ' over 5 times in the last two sections. Without anchors or strong context (e.g. a rule name), a reader is forced to scroll up and down the document trying to locate what rule ## is. For an ADR, I don't think that's a must-fix. However, we need to remember to address this in the documentation changes that we are proposing in #863. I'm not sure if that means each rule needs a formal name, we use anchors in the documents, or (maybe both) we need to make this easier to consume.
| - **Version skew becomes a supported condition.** Independently deployed services mean compatibility | ||
| matrices on customer installations, with no rollback available on a customer's own hardware. Rule | ||
| 5 keeps this tractable, and it is a permanent obligation. |
There was a problem hiding this comment.
❓ Does this allow for eventual deprecation and removal of a version ?
There was a problem hiding this comment.
That's a great question. The ADR doesn't explicitly say how new services will be versioned going forward but I will strongly advocate services living in repos of their own, with semantic version numbers evolving independently, as well, as new features are added and bugs are fixed. However, upcoming standards for internal APIs mandates services never make breaking changes which means, "the latest version of everything" is supposed to always work together; that is our obligation. (This same ADR formalizes how APIs do get formally deprecated and eventually removed.)
But, as far as versions of the services goes, a self-host customer initial install is "everything" and, to get any bug fix, the process is also the same: upgrade to the latest version of everything. In this view, every bug is forward-fix only; there are no "major versions" or release branches to maintain: just a never-ending stream of enhancements and bug fixes.
Now to your question: if you are talking about deprecating and, eventually, removing some version of some service, I would say we can do that but I am not sure it is necessary. We just continuously add features and fix bugs and, each time we do, we are publishing new versions of the services and customers can just sit on whatever version they are on or they can upgrade to the latest (of everything) to get all of the latest bug fixes across all services. From this point of view, every version of every service that is not the latest is deprecated (because it contains known bugs that have since been fixed).
Having said that, this is my "I'm new here" view of how this should work without the first conversation with Product Management so they may take a different view of how versioning this federation of services should work and may have reasons to retain the current YYYY.XX.ZZ versioning scheme.
Does that help at all?
There was a problem hiding this comment.
Thanks mike! So to clarify, you are saying that within a version there are no breaking changes to be made, but old versions can be deprecated and removed, so they do not have to be indefinitely supported?
I've added comments on the PR you link, but one of the interesting questions here is: What about security issues? Do they classify under the same category as bugs (for which breaking changes are allowed as per your linked PR)?
There was a problem hiding this comment.
Yeah - it's definitely a different way of thinking - because even sentences like this cease to have any meaning in a world of continuous deployment:
... so they do not have to be indefinitely supported?
This sentence has in mind something like release branches where a bug that is found has to be fixed in N number of releases. But if services are just continuously receiving updates, and every one results in a new version of that service, with a new semantic version number, then there is no concept of "supporting" anything other than the current version. If a bug is found in the current version or any previous version, it is just fixed on the current version - and a new version produced.
It is our job to fix any bug that is found. It is the self-host customer's job to upgrade to the latest to get whatever bugs we have fixed.
Yeah, but customers want to be able to upgrade on their own timetable. And they do not like to get new features and bug fixes at the same time. They want to just get bug fixes.
Well I guess that is a Product Management decision but, in my view, customers do not get to dictate how they get bug fixes. As long as the process to get them requires nothing more than a "update everything" script, I think that is not unreasonable.
The only scenario I can think of in a world of continuous deployment where there is a notion of "supporting" older versions of the software is if we came out with "Bitwarden - The Next Generation" which was like a completely new implementation based on a completely new code base. In this case, it would be our responsibility to fix any bug found in both code bases.
What about security issues? Do they classify under the same category as bugs (for which breaking changes are allowed as per your linked PR)?
Right. I updated the docs to clarify that any bug - security or otherwise - SHOULD be fixed "in place" - even if doing so would technically be considered a breaking change.
There was a problem hiding this comment.
I think the middle quote is from some other thread. I'm not sure where it came from?
Thanks for the response though!
There was a problem hiding this comment.
Yeah, but customers want to be able to upgrade on their own timetable. And they do not like to get new features and bug fixes at the same time. They want to just get bug fixes.
Ha! That was me trying to anticipate the objection. ;-)
There was a problem hiding this comment.
Oh, I can see why one would have that objection, but I don't hold that view at all.
It just happens to be that my team has to make a fair amount of changes to the cryptography layer, so I don't view the API as stable enough to be declared stable / am concerned what impact it will have / what mechanisms we will have to use to help with that. That's all.
There was a problem hiding this comment.
Well, if an API if a team is still actively iterating on an API (i.e. still under active development), I don't think "no breaking changes" rule applies to such an API. But, in either event, this will be more relevant to the upcoming Internal API Standards.
withinfocus
left a comment
There was a problem hiding this comment.
Few things to improve upon, potentially before merging, or they can come with the later (expected) ratification.
|
|
||
| ### Plan | ||
|
|
||
| - Publish **API Standards**. Rules 4 and 5 reference it normatively and it does not exist yet on |
There was a problem hiding this comment.
ℹ️ This will be edited / amended once in place, or its development will be clearly indicated.
Co-authored-by: Matt Bishop <matt@withinfocus.com>
…rchitecture' into docs/adr-0035-service-oriented-architecture
withinfocus
left a comment
There was a problem hiding this comment.
This works for me. Great beginnings!
| - Any cache used `MUST` be owned and invalidated by the owning service. | ||
| - Serving results from cache `MUST NOT` bypass authorization the owning service would otherwise | ||
| enforce. | ||
| 8. Services that need to read, write, or validate data owned by another service `SHOULD` do so via |
There was a problem hiding this comment.
issue: This feels like a must to me. I don't want to get in a world where we have the option to directly hit APIs to bypass guards built into a client or scatter implementation details to far-flung API call sites.
There was a problem hiding this comment.
Mike and I actually talked about this one and softened the language so as to not rule this out.
There was a problem hiding this comment.
Is there something you foresee that being beneficial for? I think it's likely to produce unspoken contracts and action at a distance effects
| 10. Services `MUST` publish events for every state change using the "transactional outbox" pattern, | ||
| regardless of whether there are any known consumers. |
There was a problem hiding this comment.
Is there value in limiting this scope initially? I may be missing things, but my read of this architecture is that only deletes are needed to propagate cascades.
Additionally, this rule binds us to broad-scoped state change statements, rather than tightly typed and defined messages. The latter requires knowledge of consumer needs, but the former creates significant challenges in potentially needing to combine data streams.
There was a problem hiding this comment.
The idea here is that if every service broadcasts every state change "as a matter of course", when there is a need to build this integration or that, the team that needs to do so has everything they need. There is never a situation where, in order to make something work, team A has to talk to team B to get them to start doing something. It is a pattern I found to be a very effective way of working.
Having said that, I think that we actually do have a consumer for all of these events - and that is our audit log (a.k.a. Events Service) which, of course, wants a permanent record of everything that's happened.
In addition to that, are the consumers you mentioned - those that want to make sure they delete related data when parent resources are deleted.
So the requirement is for those three reasons:
- To facilitate the audit log.
- To facilitate cascade deletes.
- To facilitate the next thing we haven't thought of so that, when it does come up, it doesn't require the "upstream" team to make changes to their code.
There was a problem hiding this comment.
Event logs are very far from an audit trail of everything that happens in the Bitwarden database. What's more is there's a layer of authorization that needs to occur on them and it's not obvious to me where that belongs.
I also want to point out that
To facilitate the next thing we haven't thought of so that, when it does come up, it doesn't require the "upstream" team to make changes to their code.
and
but the exact details of what is included in the event is up to the service to decide
are statements in conflict. You're trying to preempt what events will be needed without knowledge of what that may be.
My push back here is that this rule will result in generic, all-encompassing change events that are difficult to consume, annoying to maintain, and mostly ignored.
There was a problem hiding this comment.
Probably best discussed in person but I am curious to know what makes you think they would be difficult to consume? Presumably, all events share some common top-level "shape" like category, occurredAt, description, and severity which would go a long way towards making them pretty straight-forward to consume.
As for maintenance, I would expect that most transactions go through some common framework that, in addition to persisting the changes, publishes the corresponding event in which case these events all originate from one place, which shouldn't be too hard to maintain.
And if every change is something that we need to record for auditing, then, none of these events are ignored.
As for authorization, I agree that's something that needs to be worked out.
| authorization by scope, and context propagation, and build the cloud path it describes. | ||
| - Provide an Entity Framework path for organization scoping, so rule 9 holds on all supported | ||
| database providers. | ||
| - Provide a transactional outbox and a broker-free event transport, so rules 10 through 12 hold on |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Establishes the principles services must adhere to in a service-oriented architecture.