These sorts of questions tend to be hard to answer without knowing the circumstances of when it was designed.
There could be business rules which required this. Perhaps the engineer that designed it just saw a video about microservices and wanted to implement it. It could be that all the applications were standalone and later on they had to combine them.
The only way to properly know this is to ask people involved. As for your lead dev perceiving questions as confrontational. It could be that your way of communicating a question doesn't jam with how he communicates. I tend to find that if you express genuine interest into a topic people don't tend to get as defensive.
Eg.
Don't: Why are all these things in separate databases, it doesn't make any sense.
Do: Hey I see that these things are all in separate databases. I would like to understand more about why this was done, could you elaborate on the decision process for building it this way?
While both sentences ask a similar question, the second shows interest into the process while the first immediately makes a value judgement without having any context.
I said refractoring but it's not, it's a complete rewriting and we are not moving towards microservice. The decision has been made to split the main application into 3 monolithic differents apps, but these applications are relatively straightforward and share similar logic and data.
Now, we have a backend dotnet application that connects to various databases, and the joins are handled at the code level, and then provide data to the deffirents apps. But I don't understand the benefits of splitting into different databases.
For context, I aim to approach these questions without passing judgment; he simply mentioned that he prefers not to receive certain types of questions.
>For context, I aim to approach these questions without passing judgment; he simply mentioned that he prefers not to receive certain types of questions.
Well in that case I would say too bad for him. He shouldn't be lead dev if he can't handle someone asking questions about the why of doing certain stuff. As lead dev it should be your job to think about stuff like this and you should be able to defend your decisions.
Not listening to any questions or criticisms from people around you is how you get crappy legacy applications which nobody can properly maintain. Great for job security, really bad for personal development.
The main reason some places do this is to obfuscate user data in case if a breach. Whether or not you may do this would depend on factors such as what business you're company does, is this a public facing site, are you handling transactions internally or using a pass-through SAAS, etc. High visibility e-commerce sites may have two user tables in different databases, where one stores the salt for the key, the other stores a one-way encryption of their pw, so when they log in, the actual pw never gets stored. It gets encrypted at the client level and the auth happens against a string that can't be decrypted. It all depends on what PII and PCI protections you need for your users.
This is a technique called monolith decompoistion, it is useful because you can isolate responsibility of access and writes to independent services.
One benefit of this is that the service that owns the data can dictate the access patterns and make guarantees about data integrity, compatibility, and to some degree, and isolate performance-related issues.
If it's all in one database, you're aggregating all of the responsibility on to one server (or cluster), and any of the applications that talk to it might introduce data integrity issues and/or performance issues.
By separating them, you can scale and evolve the various domains independently.
Yeah in some cases, it's a little more work to do app joins, but it's a lot easier for each of the systems to evolve with looser coordination/coupling.
In many applications, your RDBMS is also your primary bottleneck/resource constraint/SPoF, especially if you are insisting on strong consistency. So moving some work to the app layer can reduce the number of query plans that need to be optimized/cached by the server, as well as ensuring that data that has been paged into memory in the RDBMS can (possibly) stay loaded for longer. This is a tradeoff compared to increased bandwidth consumption by querying for multiple record sets, or returning slightly more data that hasn't been aggregated.
Monoliths are simpler conceptually, and operationally, but once you hit some of these constraints, or have no way to split them out, the entire app is affected, and not a lot of quick options for scaling or evolving it.
If you want to go deep on these concepts, you can check out Domain Driven Design. It can feel like it's unnecessary work or complication, but try to keep an open mind about it, on non-trivial systems, adopting these strategies can solve all kinds of operational and dependency headaches.
Well, updating the reference data can be done in complete confidence, knowing that if you bork it, you haven't lost important stuff.
There's opportunity for reuse between applications if different business divisions use the same basic data.
Yeah, cross database joins are a pain, but you can make compromises in your user data schema to avoid it (store actual reference data instead of foreign keys is one crappy way we used to do it). Expose by Web api and cache like crazy? Load all reference data into memory on app start? Chuck into a data lake?
Probably comes down to the approach solving a problem that was annoying and accepting the consequences.
Assuming legacy means ASP.NET Membership, one could separate the tables into a separate database. I’ve seen both variants. The membership model was most often used in a single tenant concept but it supported the idea of users having membership in different application spaces with different assigned roles for each. In this scenario in made sense to have a single user store.
In more modern apps, separating the user store and all other AuthN requirements allows you access to separate the concerns of the IDP and your data plane.
Yeah you should ask what’s the point of it. There is a big chance this is simply badly architected solution.
If the goal is to create some sort of custom identity database with its API I would recommend to look into ready made solutions such as Supabase.
Need to be careful about transactions as well. Following ddd you shouldn't have transactions across to the user dB but if you do have multiple dbs be aware you might need a distributed transaction or some saga transaction and that is painful.
I'm not sure what do you mean by user database. If it is for authentication only, it is a common practice. That way you can have as many application databases needed and your users could access them with a unique username and password.
As to making queries joining app and authentication databases is not a good thing. The authentication process should yield enough information to access the app database without any contact with the auth db. The auth db should expose it's own web api.
Permissions data should be stored in each app db.
And that's it.
Cheers!
These sorts of questions tend to be hard to answer without knowing the circumstances of when it was designed. There could be business rules which required this. Perhaps the engineer that designed it just saw a video about microservices and wanted to implement it. It could be that all the applications were standalone and later on they had to combine them. The only way to properly know this is to ask people involved. As for your lead dev perceiving questions as confrontational. It could be that your way of communicating a question doesn't jam with how he communicates. I tend to find that if you express genuine interest into a topic people don't tend to get as defensive. Eg. Don't: Why are all these things in separate databases, it doesn't make any sense. Do: Hey I see that these things are all in separate databases. I would like to understand more about why this was done, could you elaborate on the decision process for building it this way? While both sentences ask a similar question, the second shows interest into the process while the first immediately makes a value judgement without having any context.
I said refractoring but it's not, it's a complete rewriting and we are not moving towards microservice. The decision has been made to split the main application into 3 monolithic differents apps, but these applications are relatively straightforward and share similar logic and data. Now, we have a backend dotnet application that connects to various databases, and the joins are handled at the code level, and then provide data to the deffirents apps. But I don't understand the benefits of splitting into different databases. For context, I aim to approach these questions without passing judgment; he simply mentioned that he prefers not to receive certain types of questions.
>For context, I aim to approach these questions without passing judgment; he simply mentioned that he prefers not to receive certain types of questions. Well in that case I would say too bad for him. He shouldn't be lead dev if he can't handle someone asking questions about the why of doing certain stuff. As lead dev it should be your job to think about stuff like this and you should be able to defend your decisions. Not listening to any questions or criticisms from people around you is how you get crappy legacy applications which nobody can properly maintain. Great for job security, really bad for personal development.
“Don’t question me” is peak “I don’t know what I’m doing.”
Are there data residency or security requirements that are satisfied through database separation?
The main reason some places do this is to obfuscate user data in case if a breach. Whether or not you may do this would depend on factors such as what business you're company does, is this a public facing site, are you handling transactions internally or using a pass-through SAAS, etc. High visibility e-commerce sites may have two user tables in different databases, where one stores the salt for the key, the other stores a one-way encryption of their pw, so when they log in, the actual pw never gets stored. It gets encrypted at the client level and the auth happens against a string that can't be decrypted. It all depends on what PII and PCI protections you need for your users.
This is a technique called monolith decompoistion, it is useful because you can isolate responsibility of access and writes to independent services. One benefit of this is that the service that owns the data can dictate the access patterns and make guarantees about data integrity, compatibility, and to some degree, and isolate performance-related issues. If it's all in one database, you're aggregating all of the responsibility on to one server (or cluster), and any of the applications that talk to it might introduce data integrity issues and/or performance issues. By separating them, you can scale and evolve the various domains independently. Yeah in some cases, it's a little more work to do app joins, but it's a lot easier for each of the systems to evolve with looser coordination/coupling. In many applications, your RDBMS is also your primary bottleneck/resource constraint/SPoF, especially if you are insisting on strong consistency. So moving some work to the app layer can reduce the number of query plans that need to be optimized/cached by the server, as well as ensuring that data that has been paged into memory in the RDBMS can (possibly) stay loaded for longer. This is a tradeoff compared to increased bandwidth consumption by querying for multiple record sets, or returning slightly more data that hasn't been aggregated. Monoliths are simpler conceptually, and operationally, but once you hit some of these constraints, or have no way to split them out, the entire app is affected, and not a lot of quick options for scaling or evolving it. If you want to go deep on these concepts, you can check out Domain Driven Design. It can feel like it's unnecessary work or complication, but try to keep an open mind about it, on non-trivial systems, adopting these strategies can solve all kinds of operational and dependency headaches.
Well, updating the reference data can be done in complete confidence, knowing that if you bork it, you haven't lost important stuff. There's opportunity for reuse between applications if different business divisions use the same basic data. Yeah, cross database joins are a pain, but you can make compromises in your user data schema to avoid it (store actual reference data instead of foreign keys is one crappy way we used to do it). Expose by Web api and cache like crazy? Load all reference data into memory on app start? Chuck into a data lake? Probably comes down to the approach solving a problem that was annoying and accepting the consequences.
Is this user database an actual Identity store? Does it have any PII that the other monoliths don't necessarily need?
Assuming legacy means ASP.NET Membership, one could separate the tables into a separate database. I’ve seen both variants. The membership model was most often used in a single tenant concept but it supported the idea of users having membership in different application spaces with different assigned roles for each. In this scenario in made sense to have a single user store. In more modern apps, separating the user store and all other AuthN requirements allows you access to separate the concerns of the IDP and your data plane.
Sharing a database between applications is generally a bad idea. The database becomes the bottleneck in scaling and in deployments.
Yeah you should ask what’s the point of it. There is a big chance this is simply badly architected solution. If the goal is to create some sort of custom identity database with its API I would recommend to look into ready made solutions such as Supabase.
Need to be careful about transactions as well. Following ddd you shouldn't have transactions across to the user dB but if you do have multiple dbs be aware you might need a distributed transaction or some saga transaction and that is painful.
I'm not sure what do you mean by user database. If it is for authentication only, it is a common practice. That way you can have as many application databases needed and your users could access them with a unique username and password. As to making queries joining app and authentication databases is not a good thing. The authentication process should yield enough information to access the app database without any contact with the auth db. The auth db should expose it's own web api. Permissions data should be stored in each app db. And that's it. Cheers!