Businesses are building digital platforms with modern architecture principles like domain-driven, microservice-based, and event-driven. These platforms are getting ever so modular, flexible, and complex.
While they are built with architecture principles like - loose coupling, individually scaling, plug-and-play components; regulations and security considerations on data - this complexity leads to many unknown and grey areas in the entire architecture. Details on how the different components of this complex architecture interact with each other are lost. Generating insights becomes multi-teams, multi-staged activity and hence multi-days activity. Manual analysis and actions on these insights takes even more time, making the business less agile.
We will share how we made all the business and technical insights of a complicated platform available in real-time with little incremental effort and constant validation of the ideas and slices with business teams.
We kept the self-service aspect using configurability and scalability, at the center of our solution - to accommodate increasing components in the source platform, evolving requirements, and even supporting new platforms altogether. This led to evolving the solution in lines of domain data products, where the data is generated and consumed by those who understand it the best.
We will discuss our journey toward data mesh-like architecture and domain data boundaries, domain thinking that helped us in evolving/maintaining a data platform.
Enterprises are large in operations, geographical spread, and employee spread. Serving everyone is a challenge faced by every organization. At the same time, we have also heard stories of well-built software not getting adopted due to resistance to usage or change.
Today, the platform we built has evolved to -
- Generate business insights that are converted into nudges for business growth (Automated intelligence delivering business value).
- Empower business and data teams to work with business data to make faster decisions interactively (Impact of Data Intelligence in Software Development Life Cycle).
- Provide an environment where data analysts and domain teams can uncover hidden patterns and unknown unknowns (Impact of Data Intelligence in Software Development Life Cycle).
- Enable application teams to consume data platforms for ingestion and extraction as self-service. Essentially, the data platform has become an infrastructure for application teams to derive insights into their platform health and business teams for business health.
