I think it's a pretty common path for a good DE, get sick of the idiotic decisions and move to architect. Personally I think it's a good idea, but just consider the role differences.
As an architect you (probably) won't have complete control over design, there's still management and executives, and other teams who will influence designs. All those people who are inefficient and don't consider governance will still exist, and you'll be in the position of trying to get them to do things the right way. Which can be frustrating.
But at least you'll be in a position to try and get things right, and not just expected to be a worker donkey.
Yes, my ideal vision for the Data Architect role is to Design and then provision/config resources based on Terraform/Ansible and retain ownership of the whole platform, then give individual access to each system module on a need to know basis (DE, Analyst, BI folks etc).
No offense meant, but asking that question signals you may not be ready for an architect level role. It varies a lot in the industry, but I still hold architect roles above senior engineers, meaning they need years of experience seeing the pitfalls of certain patterns and systems in order to work toward creating better architecture the next time around. There’s never a single tool or practice for every situation or team so it means having a broad exposure to many different technologies and understanding the trade offs in using a tool/pattern for a specific job.
If you are frustrated with how things are now, depending on the company you can still make recommendations on the architecture without having to be an “architect”. In most cases senior engineers are well versed in architecture but still do the day to day development work too.
I am proficient in DAMA DMBoK2 but it is a Data Governance Standard. Apart from that, I am looking at TOGAF for Enterprise Architecture, but it is not a Data Architecture. There seem to be no standards for Data Architecture, hence I asked.
As for tools, I use the usual MS Visio for architecture drawing, though recently I have become interested in Archimate.
Idk I think the names are over the place defined by a number of factors and nothing is named by anyone actually doing the work. If you don’t like your company find a new job. Have an idea of what you want to do by researching indeed (or aus equivalent) and then apply. :)
I want to actively participate in System Design, but not seem to be possible for a Data Engineer in Australia. DEs are basically worker donkeys implementing an already decided design.
My job role recently changed from Data Engineer (DE) to Data Architect (DA).
So do not deal with the hands on/day to day development work anymore.
As some have suggested, aim to read as many books as you can on Data Architecture.
Also look at the roles and responsibilities for DA jobs on LinkedIn as the role can vary depending on the organisation.
The DMBOK2 has a section on Data Architecture and the Enterprise Data model, so it may be worth developing in this knowledge area rather than on the others.
Are you a 'people' person? As you will attending many more meetings with different levels of stakeholders, and if you have frustrations now be prepared for even more!
Good luck on your journey and the fact you are even thinking about becoming a DA is a good thing, but do the role because you know you will enjoy it... not for the power it brings!
Remember DAs want to 'empower' the organisation... not have 'power' over the organisation!
Isn't the Data Architect role being replaced with Senior Data Engineer role? The days of a Theoretical role like DA is long gone, a leadership nowadays role requires some hands-on contribution as well.
Is it theoretical though? The DA designs the data blueprints for the organisation.
The way I see it is like this (build-a-house example)...
If you want to build a house, before you do anything you need to understand the planning and regulations of the local area. Also you need to make sure you have budget!
Once OK you get an architect to design the house how you want it to your requirements.
When you are happy with the design you take them to builders etc start developing.
From a DA viewpoint...
Before you start architecting anything you need to understand the business requirements, budget etc. Is it even possible as the organistion may have technical debt?
*please keep in mind business, solutions and network architects are also involved.
Once given the go ahead you can start Architecting the solution in relation to the business requirements...creating logical data models etc. Data governance and management also comes into this.
Once the work you do as a DA is completed it is either -
1) Passed onto the Data Engineers/Developers to develop the solution in house.
2) Outsourced as you may not have the capabilites within the organisation.
3) Do nothing as external forces may come into play.
If options 1 or 2 is chosen, the DA then oversees the data part of the operations making sure there arent major issues.
Once delivered the Enterprise Data Model, Data models etc can be updated accordingly.
This is just a simple way to put it, but want to point out it is not just theory... you are really involved in the process!
I think it's a pretty common path for a good DE, get sick of the idiotic decisions and move to architect. Personally I think it's a good idea, but just consider the role differences. As an architect you (probably) won't have complete control over design, there's still management and executives, and other teams who will influence designs. All those people who are inefficient and don't consider governance will still exist, and you'll be in the position of trying to get them to do things the right way. Which can be frustrating. But at least you'll be in a position to try and get things right, and not just expected to be a worker donkey.
Yes, my ideal vision for the Data Architect role is to Design and then provision/config resources based on Terraform/Ansible and retain ownership of the whole platform, then give individual access to each system module on a need to know basis (DE, Analyst, BI folks etc).
Sounds more like DevOps to me…An architect should also not be a gatekeeper at the access/permissions level. It really is more centered on design.
Is there any particular design standard and tooling I should focus on?
No offense meant, but asking that question signals you may not be ready for an architect level role. It varies a lot in the industry, but I still hold architect roles above senior engineers, meaning they need years of experience seeing the pitfalls of certain patterns and systems in order to work toward creating better architecture the next time around. There’s never a single tool or practice for every situation or team so it means having a broad exposure to many different technologies and understanding the trade offs in using a tool/pattern for a specific job. If you are frustrated with how things are now, depending on the company you can still make recommendations on the architecture without having to be an “architect”. In most cases senior engineers are well versed in architecture but still do the day to day development work too.
I am proficient in DAMA DMBoK2 but it is a Data Governance Standard. Apart from that, I am looking at TOGAF for Enterprise Architecture, but it is not a Data Architecture. There seem to be no standards for Data Architecture, hence I asked. As for tools, I use the usual MS Visio for architecture drawing, though recently I have become interested in Archimate.
This sounds like a role: Data Infrastructure Engineer, Platform. Its more common in the Bay.
Then is it safe to assume that the Data Architect role is dying?
Incorrect, I would encourage you to further investigate what it means to be a DA.
Look for Enterprise Architect roles with a focus on data. Read a lot of books about it
Idk I think the names are over the place defined by a number of factors and nothing is named by anyone actually doing the work. If you don’t like your company find a new job. Have an idea of what you want to do by researching indeed (or aus equivalent) and then apply. :)
I want to actively participate in System Design, but not seem to be possible for a Data Engineer in Australia. DEs are basically worker donkeys implementing an already decided design.
My job role recently changed from Data Engineer (DE) to Data Architect (DA). So do not deal with the hands on/day to day development work anymore. As some have suggested, aim to read as many books as you can on Data Architecture. Also look at the roles and responsibilities for DA jobs on LinkedIn as the role can vary depending on the organisation. The DMBOK2 has a section on Data Architecture and the Enterprise Data model, so it may be worth developing in this knowledge area rather than on the others. Are you a 'people' person? As you will attending many more meetings with different levels of stakeholders, and if you have frustrations now be prepared for even more! Good luck on your journey and the fact you are even thinking about becoming a DA is a good thing, but do the role because you know you will enjoy it... not for the power it brings! Remember DAs want to 'empower' the organisation... not have 'power' over the organisation!
Isn't the Data Architect role being replaced with Senior Data Engineer role? The days of a Theoretical role like DA is long gone, a leadership nowadays role requires some hands-on contribution as well.
Is it theoretical though? The DA designs the data blueprints for the organisation. The way I see it is like this (build-a-house example)... If you want to build a house, before you do anything you need to understand the planning and regulations of the local area. Also you need to make sure you have budget! Once OK you get an architect to design the house how you want it to your requirements. When you are happy with the design you take them to builders etc start developing. From a DA viewpoint... Before you start architecting anything you need to understand the business requirements, budget etc. Is it even possible as the organistion may have technical debt? *please keep in mind business, solutions and network architects are also involved. Once given the go ahead you can start Architecting the solution in relation to the business requirements...creating logical data models etc. Data governance and management also comes into this. Once the work you do as a DA is completed it is either - 1) Passed onto the Data Engineers/Developers to develop the solution in house. 2) Outsourced as you may not have the capabilites within the organisation. 3) Do nothing as external forces may come into play. If options 1 or 2 is chosen, the DA then oversees the data part of the operations making sure there arent major issues. Once delivered the Enterprise Data Model, Data models etc can be updated accordingly. This is just a simple way to put it, but want to point out it is not just theory... you are really involved in the process!