🔢Pearl Zhu - Collected Quotes

"A good strategy tells you not only what specifically needs to accomplish, but WHY." (Pearl Zhu, "Digitizing Boardroom: The Multifaceted Aspects of Digital Ready Boards", 2016)

"Agile is more a 'direction', than an 'end'. Transforming to Agile culture means the business knows the direction they want to go on." (Pearl Zhu, "Digital Agility: The Rocky Road from Doing Agile to Being Agile", 2016)

"Breaking rules is indeed an important part of creativity. Innovation needs a level of guidance." (Pearl Zhu,  "Digitizing Boardroom: The Multifaceted Aspects of Digital Ready Boards", 2016)

"Good governance is less about structure and rules than being focused, effective and accountable." (Pearl Zhu,  "Digitizing Boardroom: The Multifaceted Aspects of Digital Ready Boards", 2016)

"Governance is not about maximization, but about optimization." (Pearl Zhu, "Digitizing Boardroom: The Multifaceted Aspects of Digital Ready Boards", 2016)

"Selecting the right measure and measuring things right are both art and science. And KPIs influence management behavior as well as business culture." (Pearl Zhu, "CIO Master: Unleash the Digital Potential of It", 2016)

"Setting the right priorities or having superior time management skill means knowing the difference between 'must have', and 'nice to have'." (Pearl Zhu, "Thinkingaire: 100 Game Changing Digital Mindsets to Compete for the Future", 2016)

"The art of questioning is to ignite innovative thinking; the science of questioning is to frame system thinking, with the progressive pursuit of better solutions." (Pearl Zhu, "Leadership Master: Five Digital Trends to Leap Leadership Maturity", 2016)

"The 'result' of micromanagement is perhaps tangible in the short run, but more often causes damage for the long term." (Pearl Zhu, "Change Insight: Change as an Ongoing Capability to Fuel Digital Transformation", 2016)

"Using two-dimensional lenses to perceive the multi-faceted world can limit your ability to observe the world more objectively." (Pearl Zhu, "Thinkingaire: 100 Game Changing Digital Mindsets to Compete for the Future", 2016)

"A performance dashboard is a practical tool to improve management effectiveness and efficiency, not just a pretty retrospective picture in an annual report." (Pearl Zhu, "Performance Master: Take a Holistic Approach to Unlock Digital Performance", 2017)

"A 'roadmap' is simply a plan for moving or transitioning, from one state to another. A roadmap provides the direction to the future." (Pearl Zhu, "Digital Capability: Building Lego Like Capability Into Business Competency", 2017)

"A well-defined set of digital rules are not for limiting innovation, but for setting the frame of relevance and guide through changes and digital transformation." (Pearl Zhu, "100 Digital Rules: Setting Guidelines to Explore Digital New Normal", 2017)

"Building a comprehensive problem-solving framework is about leveraging a structured methodology that allows you to frame problems systematically and solve problems creatively." (Pearl Zhu, "Problem Solving Master: Frame Problems Systematically and Solve Problem Creatively", 2017)

"Decision makers with emotional excellence have the ability to dispassionately examine alternatives via fact finding, analysis, structured planning, objective evaluations, and comparison." (Pearl Zhu, "Decision Master: The Art and Science of Decision Making", 2017)

"Decision making is an art only until the person understands the science." (Pearl Zhu, "Decision Master: The Art and Science of Decision Making", 2017)

"Decision maturity is to ensure the right decisions have been made by the right people at the right time to solve the right problems." (Pearl Zhu, "Decision Master: The Art and Science of Decision Making", 2017)

"Digital synchronization and strategic alignment occur when all parts of the choir sing their respective parts in harmony to achieve a higher purpose." (Pearl Zhu, "12 CIO Personas: The Digital CIO's Situational Leadership Practices", 2017)

"Digitalization implies the full-scale changes in the way business is conducted so that it’s a multi-dimensional planning and orchestration." (Pearl Zhu, "Digital Capability: Building Lego Like Capability Into Business Competency", 2017)

"Framing the right problem is equally or even more important than solving it." (Pearl Zhu, “Change, Creativity and Problem-Solving”, 2017)

"Most organizations fail to manage performance effectively because they fail to look into the system holistically." (Pearl Zhu, "Performance Master: Take a Holistic Approach to Unlock Digital Performance", 2017)

"The science of decision-making is to make sure there is an effective decision process in place." (Pearl Zhu, "Decision Master: The Art and Science of Decision Making", 2017)

"It is important to strengthen the weakest link, to ensure all important business elements integrated and knitted into ongoing organizational capabilities and unique business competency." (Pearl Zhu, "Digital Capability: Building Lego Like Capability Into Business Competency", 2017)

"The simplicity and the complexity are just the opposite ends of the same spectrum." (Pearl Zhu, "Digital Gaps: Bridging Multiple Gaps to Run Cohesive Digital Business", 2017)

"We are moving slowly into an era where Big Data is the starting point, not the end." (Pearl Zhu, "Digital Master: Debunk the Myths of Enterprise Digital Maturity", 2017)

"You can’t improve what you are not managing, you can’t manage what you are not measuring, and you can’t measure what you are not focusing." (Pearl Zhu, "Digital Capability: Building Lego Like Capability Into Business Competency", 2017)

"A business ecosystem is just like the natural ecosystem; first, needs to be understood, then, needs to be well planned, and also needs to be thoughtfully renewed as well." (Pearl Zhu, "Digital Maturity: Take a Journey of a Thousand Miles from Functioning to Delight", 2018)

"A seamless digital transformation requires a vision to convey 'WHY', a solid strategy to clarify 'WHAT', and a technical specification to articulate 'HOW' you want to transform radically." (Pearl Zhu, "Digital Maturity: Take a Journey of a Thousand Miles from Functioning to Delight", 2018)

"An organizational structure carries inherent capabilities as to what can be achieved within its frame." (Pearl Zhu, Digital Maturity: Take a Journey of a Thousand Miles from Functioning to Delight, 2018)

"Change Management is a journey, not just a one-time project, riding ahead of change curve takes both strategy and methodology." (Pearl Zhu, "The Change Agent CIO: The CIO’s Dynamic Role of Leading Digitalization", 2018)

"Coherence improves business flow; resilience makes business robust and anti-fragile." (Pearl Zhu, "Digital Hybridity: How to Strike the Right Balance for Digital Paradigm Shift", 2018)

"Going digital is more like a journey than a destination. Predicting and preparing the next level of digitalization is an iterative learning and doing continuum." (Pearl Zhu, "Digital Maturity: Take a Journey of a Thousand Miles from Functioning to Delight", 2018)

"Ideally, the two structures - hierarchy, and relationship structure wrap around each other to ensure responsibility, to keep information flow and the creation of power." (Pearl Zhu, "Digital Maturity: Take a Journey of a Thousand Miles from Functioning to Delight", 2018)

"Taking the multidimensional hybrid models for going digital is all about how to strike the right balance of reaping quick wins and focusing on the long-term strategic goals." (Pearl Zhu, "Digital Hybridity: How to Strike the Right Balance for Digital Paradigm Shift", 2018)

"The most effective digital workplace is one where collaboration and sharing are the norms." (Pearl Zhu, "Digital Maturity: Take a Journey of a Thousand Miles from Functioning to Delight", 2018)

🎯Rukmani Gopalan - Collected Quotes

"A cloud data warehouse is an enterprise data warehouse offered as a managed service (PaaS) on public clouds with optimized integrations for data ingestion, analytics processing, and BI analytics." (Rukmani Gopalan, "The Cloud Data Lake: A Guide to Building Robust Cloud Data Architecture", 2022) 

"Churn refers to rapidly changing the activities and your plan when they are in flux - this is disruptive to your organization and slows your progress. Change refers to an inevitable movement in requirements and helps you plan for and execute this movement thoughtfully." (Rukmani Gopalan, "The Cloud Data Lake: A Guide to Building Robust Cloud Data Architecture", 2022) 

"Data mesh relies on a distributed architecture that consists of domains. Each domain is an independent unit of data and its associated storage and compute components. When an organization contains various product units, each with its own data needs, each product team owns a domain that is operated and governed independently by the product team. […] Data mesh has a unique value proposition, not just offering scale of infrastructure and scenarios but also helping shift the organization’s culture around data," (Rukmani Gopalan, "The Cloud Data Lake: A Guide to Building Robust Cloud Data Architecture", 2022)

"If there is one thing I strongly recommend, it is to invest in a cloud data lake and start collecting and processing data that you believe is useful to your organization today." (Rukmani Gopalan, "The Cloud Data Lake: A Guide to Building Robust Cloud Data Architecture", 2022) 

"It’s true that data and data strategy are critical to the organization; however, it’s also true that data by itself is a means to the end of business or customer impact unless you’re a provider of data or data-related services." (Rukmani Gopalan, "The Cloud Data Lake: A Guide to Building Robust Cloud Data Architecture", 2022) 

"Plan for customer impact, and prepare to learn and fine-tune as you progress. Make choices based on the impact they offer to customers, and stay consistent in your implementation while keeping open-minded for learnings. Especially if you are an early adopter of a technology, you can help develop the technology with the provider and thus get ample support from the technology provider in return. Similarly, identify highly motivated early adopters within your customer base and offer to develop your solution with them." (Rukmani Gopalan, "The Cloud Data Lake: A Guide to Building Robust Cloud Data Architecture", 2022) 

"Real-time stream processing refers to the ingestion, processing, and consumption of data with a specific focus on speed, targeting near real time - that is, almost instantaneous results. […] Real-time stream processing pipelines involve data that is arriving from its source at very high velocity; in other words, it is data that is streaming into the system, just like rain or a waterfall." (Rukmani Gopalan, "The Cloud Data Lake: A Guide to Building Robust Cloud Data Architecture", 2022) 

"The lakehouse provides a key advantage over the modern data warehouse by eliminating the need to have two places to store the same data. [...] Data lakehouses offer the key benefit of being able to run performant BI/SQL-based scenarios directly on the data lake, right alongside the other exploratory data science and machine learning scenarios." (Rukmani Gopalan, "The Cloud Data Lake: A Guide to Building Robust Cloud Data Architecture", 2022) 

"The promise of a cloud data lake architecture lies in the boundless diversity of scenarios that it enables." (Rukmani Gopalan, "The Cloud Data Lake: A Guide to Building Robust Cloud Data Architecture", 2022) 

"The very simple definition of cloud data lake storage is a service available as a cloud offering that can serve as a central repository for all kinds of data (structured, unstructured, and semistructured) and can support data and transactions at a large scale." (Rukmani Gopalan, "The Cloud Data Lake: A Guide to Building Robust Cloud Data Architecture", 2022)

"When it comes to data lakes, some things usually stay constant: the storage and processing patterns. Change could come in any of the following ways: Adding new components and processing or consumption patterns to respond to new requirements. […] Optimizing existing architecture for better cost or performance" (Rukmani Gopalan, "The Cloud Data Lake: A Guide to Building Robust Cloud Data Architecture", 2022)

🔢Robert Hawker - Collected Quotes

"[...] a conceptual data model [...] is system-agnostic and is a diagrammatic business representation of how different types of data are associated with one another in the organization." (Robert Hawker, "Practical Data Quality", 2023)

"A data quality rule is logic that is applied to each row of a dataset, which can determine whether the row of data is correct or incorrect. Correct data is deemed to have passed the rule, and incorrect data is deemed to have failed the rule – hence, the term failed data [...]" (Robert Hawker, "Practical Data Quality", 2023)

"Correction of data in the secondary source is not recommended. However, it is important to recognize that sometimes, secondary source fixes are required." (Robert Hawker, "Practical Data Quality", 2023)

"Data discovery is the process where an organization obtains an understanding of which data matters the most and identifies challenges with that data. The outcome of data discovery is that the scope of a data quality initiative should be clear and data quality rules can be defined." (Robert Hawker, "Practical Data Quality", 2023)

"Data profiling assesses a set of data and provides information on the values, the length of strings, the level of completeness, and the distribution patterns of each column." (Robert Hawker, "Practical Data Quality", 2023)

"Data quality rules are only effective if they are tightly scoped. Generic rules tend to produce a lot of unwanted failed records, and business users start to ignore the results. Once business users lose faith in what they see from a data quality tool, it is hard to restore engagement." (Robert Hawker, "Practical Data Quality", 2023)

"Every data quality initiative is different, and senior stakeholders at different organizations will have different needs." (Robert Hawker, "Practical Data Quality", 2023)

"If an organization had a single overall data quality key performance indicator (KPI), then it might be appropriate to put a greater weighting on those rules which would impact regulatory compliance. A lack of regulatory compliance is a risk to the very existence of organizations like these, and therefore, a greater weighting might be needed." (Robert Hawker, "Practical Data Quality", 2023)

"It rarely makes sense to aim for what people might consider perfect data (every record is complete, accurate, and up to date). The investment required is usually prohibitive, and the gains made for the last 1% of data quality improvement effort become far too marginal." (Robert Hawker, "Practical Data Quality", 2023)

"In truth, no one knows how much bad data quality costs a company – even companies with mature data quality initiatives in place, who are measuring hundreds of data points for their quality struggle to accurately measure quantitative impact. This is often a deal-breaker for senior leaders when trying to get approval for a budget for data quality work. Data quality initiatives often seek substantial budgets and are up against projects with more tangible benefits." (Robert Hawker, "Practical Data Quality", 2023)

"Momentum is important in data quality initiatives. If an issue is problematic, even where the priority is high, it can be better to move on to an issue that can be progressed efficiently." (Robert Hawker, "Practical Data Quality", 2023)

"Most data quality issues will re-occur if the root cause is not fully understood [...]" (Robert Hawker, "Practical Data Quality", 2023)

"Organizations will always only have a limited amount of resources available to remediate data. It will almost certainly not be possible to tackle all the issues at the same time. Therefore, prioritization is key to ensuring that the most value is generated from the available resources." (Robert Hawker, "Practical Data Quality", 2023)

"Successful organizations try to put a holistic data culture in place. Everyone is educated on the basics of looking after data and the importance of having good data. They consider what they have learned when performing their day-to-day tasks. This is often referred to as the promotion of good data literacy." (Robert Hawker, "Practical Data Quality", 2023)

"The biggest mistake that can be made in a data quality initiative is focusing on the wrong data. If you fix data that does not impact a critical business process or drive important decisions, your initiative simply will not make the difference that you want it to." (Robert Hawker, "Practical Data Quality", 2023)

"The data should be monitored in the source, it should be corrected in the source, and it should then feed the secondary source(s) with high-quality data that can be used without workarounds. The reduction in workarounds will make the data engineers, scientists, and data visualization specialists much more productive." (Robert Hawker, "Practical Data Quality", 2023)

"The level of data quality in an organization is the extent to which data can be used for its intended purposes."  (Robert Hawker, "Practical Data Quality", 2023)

"Start with a business strategy. Too many organizations start their data quality initiative by looking at the details of the data and trying to see 'what is wrong with it'. The right approach is to understand what the business is trying to achieve and to work out where data issues might impede this. It ensures that data quality work will be truly impactful." (Robert Hawker, "Practical Data Quality", 2023)

🎯Hubert Dulay - Collected Quotes

"A data fabric is a pattern that is very similar to a data mesh in that both provide solutions encompassing data governance and self-service: discovery, access, security, integration, transformation, and lineage. [...] In simple terms, a data fabric is a metadriven means of connecting disparate sets of data and related tools to provide a cohesive data experience and to deliver data in a self-service manner." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"A data fabric is an architectural approach to provide data access across multiple technologies and platforms, and is based on a technology solution. One key contrast is that a data mesh is much more than just technology: it is a pattern that involves people and processes. Instead of taking ownership of an entire data platform, as in a data fabric, the data mesh allows data producers to focus on data production, allows data consumers to focus on consumption, and allows hybrid teams to consume other data products, blend other data to create even more interesting data products, and publish these data products - with some data governance considerations in place." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"A domain has two main roles: data product engineer (or just data engineer) and the data product owner (or data product manager, or data steward). These roles can be the same or dedicated people in the domain. Data product owners must have a deep understanding of who their data consumers are, how the data is used, and what methods are used to consume the data. This will help ensure that the data products meet the needs of their use cases. Data product engineers are responsible for creating data products that are high quality, reliable, and usable by consumers. It should be possible to extend existing domain roles to include these domain roles with minimal effort." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"Consumability is a very important requirement because it will directly affect the experience domain consumers will have in a streaming data mesh. If other domains cannot easily consume streaming data products, then they may opt out of the streaming data mesh and decide to build their own integrations by hand, bypassing any issues they encounter with the data mesh. Some factors to consider when ingesting data derivatives that will affect the consumability of other domains are as follows: (*) Lack of scalability (*) Lack of interoperability" (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"Data governance creates access controls between the data product producer and consumer and provides metadata like schema definitions and lineages. In some cases, mastered data along with reference data may be relevant to the implementation. Data governance allows us to create appropriate access controls for these resources as well." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"Data governance is a set of policies, standards, processes, roles, and responsibilities that collectively ensure accountability and ownership of data across the business. Policies are the rules and regulations surrounding data defined by the business itself or, more importantly, externally by laws that, if broken, could cost a business a massive amount in fines. These policies also include enforcement of standards that enable interoperability and consumability of data between domains, especially in a decentralized data platform like a streaming data mesh. These policies are implemented as processes and controls on data by authorizing, authenticating, and safeguarding private or personal data. Policies are implemented using roles that represent groups, people, or systems to create access controls around data." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"Data lineage is the path the data took from its source origin, the stops it made along the way, and its destination. This includes information on all the systems it passed through, how it was cleansed, what it was enriched with, and how it was secured. Capturing all that metadata is difficult because many of those systems and applications don’t share information. It’s up to you to assemble the data’s path by pulling metadata from all those systems/applications and assembling them in hopes that you find the path your data took from its current location (destination) to its source system. Lineage is probably the hardest piece of metadata to acquire for either streaming or batching data pipelines." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"Data lineage provides the entire history of the data: origin, transformations, enrichments, users who engineered the transformations, etc. Consumers of the data need to trust that the data they will be using is the correct data. Data lineage provides a perspective that creates trust. It does so by mapping out the steps for policies, standards, processes, roles, and responsibilities that were involved with the sourcing, transformation, enrichment, and cleansing of the data." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"Data mesh is not completely decentralized. The data is decentralized in domains, but the mesh part of data mesh is not. Data governance is critical in building the mesh in a data mesh." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"Data tags are a simple way of providing the consuming domains with more information about the streaming data product: how it was built and what to expect when consuming it. Many of the streaming data characteristics are hard to measure, like quality and security, so it’s sometimes tough to provide that important information to the consuming domain. Instead of providing a number or a score, we can provide tags that represent levels of quality and security." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"Domain-driven design (DDD) is the methodology that helps us understand complex domain models by connecting the data model itself to core business concepts. The understanding that emerges from DDD creates a foundation to designing distributed, microservice-based, client-facing applications. DDD connects the implementation of software and its components to an evolving and ever-changing data model. The domain is the world of the business you are working with and the problems you are trying to solve. This typically involves rules, processes, and existing systems that need to be integrated as part of your solution." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"In a data mesh, data is decentralized, while in a data fabric, centralization of data is allowed. And with data centralization like data lakes, you get the monolithic problems that come with it. Data mesh tries to apply a microservices approach to data by decomposing data domains into smaller and more agile groups." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"Since domains are used to create data products, and sharing data products across many domains ultimately builds a mesh of data, we need to ensure that the data being served follows some guidelines. Data governance involves creating and adhering to a set of global rules, standards, and policies applied to all data products and their interfaces to ensure a collaborative and interoperable data mesh community. These guidelines must be agreed upon among the participating data mesh domains." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"The best approach to building self-services for domains is to follow what many SaaS services do. They follow a serverless model that is easier for their users to understand and utilize in their applications. The intention of the serverless SaaS providers is to not require their users to worry about the 'servers' that are allocated on their behalf. Users can focus more on their business rather than managing and tuning servers. This should be the same model for self-services: to make a streaming data mesh serverless so domains need to focus only on their business."(Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"To overcome ambiguous domain challenges, each domain boundary must be distinct and explicit. Business area, processes, and data that belong together need to stay together. Additionally, each data domain should belong to one, and only one, Agile or DevOps team. Data integration points within a data domain should be manageable and understood by all team members." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"We recommend making domain boundaries concrete and immutable. This helps avoid lengthy discussions about who owns what data, and also prohibits teams from freely interpreting domain boundaries to suit their own needs. Creating a domain-oriented structure is a transition - not only for data, but for people and resources. When creating domain boundaries, resources may eventually align with other teams, disrupting and evolving the current team structure. The entire concept of data mesh is just as much about resource alignment as it is about data, so the realignment of resources should not be considered a roadblock as you go through this process." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"When building a data mesh, it is necessary to enable existing engineers in a domain to perform the tasks required. Domains have to capture data from their operational stores, transform (join or enrich, aggregate, balance) that data, and publish their data products to the data mesh. Self-service services are the “easy buttons” necessary to make data mesh easy to adopt with high usability. In summary, the selfservices enable the domain engineers to take on many of the tasks the data engineer was responsible for across all lines of the business. A data mesh not only breaks up the monolithic data lake, but also breaks up the monolithic role of the data engineer into simple tasks the domain engineers can perform." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"While a data mesh seeks to solve many of the same problems that a data fabric addresses - namely, the ability to address data in a single, composite data environment—the approach is different. While a data fabric enables users to create a single, virtual layer on top of distributed data, a data mesh further empowers distributed groups of data producers to manage and publish data as they see fit. Data fabrics allow for a low-to-no-code data virtualization experience by applying data integration within APIs that reside within the data fabric. The data mesh, however, allows for data engineers to write code for APIs with which to interface further. Without clearly defined boundaries, domains appear to be too interconnected, and ownership becomes either political or subject to interpretation. For instance, a large retailer most likely has multiple domains. [...]" (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

🎯Anindita Mahapatra - Collected Quotes

"A data pipeline is an artifact of a data engineering process. It transforms raw data into data ready for analytics. These in turn help solve problems, aid support decisions, and make our lives more convenient. In some ways, it can be thought of as the stitch between the OLTP and OLAP systems. Data pipelines are sometimes referred to as ETL, which stands for extract, transform, load, and it has a variation called extract, load, transform (ELT). The main difference between the two is whether the incoming data is first saved to disk and then transformed (data wrangling) or vice versa. The processing is loosely referred to as ETL. Although, it is fair to say ELT is relevant in the context of Data Lakes and unstructured data, whereas ETL is used for Data Warehouses." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"A data silo is an isolated source of data that is only accessible to a single line of business (LOB) or department. It leads to inefficiencies, wasted resources, and obstacles in the form of incomplete data profiles and the inability to construct deep insights. [...] On the other hand, a data swamp is a large body of data that is ungoverned and unreliable. It is hard to find data and even harder to use it, which is why it's often used out of context. This is the opposite of data silos in the sense that the data is there and has been brought together, but because it has been done without adequate process and policy, it is as good as not being there. That would be a wasted investment." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"A model that has made it into production is a wonderful achievement! However, the journey does not stop there. There is a whole separate pipeline around model management. Over time, the model becomes stale and needs to be retrained. Yet another separate pipeline to monitor drift is needed. Model drift is often on account of data drift and is a signal to trigger a retraining process. This is where the champion model in production is compared against a new challenger version to see whether it is time to be replaced or not. Over time, it is important to be able to query what version exists in production, so that there is no confusion about which is the active one, which is the challenger, and which one needs to be promoted or rolled back. Many people have no idea what version is in production! This is where a central model registry that serves as the single source of truth for the models and their stages and versions is imperative." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"Data-driven organizations exhibit a culture of analytics. This cannot be confined to just a few premiere groups but rather to the entire organization. There are both cultural and technical challenges to overcome and this is where people, processes, and tools need to come together to bring around sustainable changes. Every business needs a strategy for business transformation." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"Data engineering is the process of converting raw data into analytics-ready data that is more accessible, usable, and consumable than its raw format. Modern companies are increasingly becoming data-driven, which means they use data to make business decisions to give them better insights into their customers and business operations. They can use these to improve profitability, reduce costs, and give them a competitive edge in the market. Behind the scenes, a series of tasks and processes are performed by a host of data personas who build reliable pipelines to source, transform, and analyze data so that it is a repeatable and mostly automated process." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"Data governance refers to aligning all aspects of data strategy, business strategy, and compliance requirements. A three-pronged approach of people, policy, and process will provide oversight for all data operations from the time data touches a system to the point it leaves. Roles and responsibilities dictate who has access to what data, something that needs to be enforced and monitored. Data lineage is tracked to provide accountability for how data has been transformed at various steps. Delta's history functionality provides a good audit trail. A central catalog builds on top of it and provides a central place for defining the rules, enforcing them, and monitoring compliance via audit logs. Some of these catalogs have to be built and stitched together unless a managed platform that has taken care of these aspects is leveraged." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"Data lakes have been in existence for a while now, so their need is no longer questioned. What is more relevant is the specifics of the solution's implementation. Consolidating all the siloed data by itself does not constitute a data lake. However, it is a starting point. Layering in governance makes the data consumable and is a step toward a curated data lake. Big data systems provide scale out of the box but force us to make some accommodations for data quality. Age-old aspects of transactional integrity were compromised on a distributed system because it was very hard to maintain ACID compliance. Due to this, BASE properties were favored. All of this was moving the needle in the wrong direction and from pristine data lakes we were moving toward data swamps, where the data could not be trusted and hence insights that were generated on the data could not be trusted either." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"Lakehouse is a new architecture and data storage paradigm that combines the characteristics of both data warehouses and data lakes to create a unified basis for all types of use cases to be built on top of it. There is no need to move data around. Data is curated and remains in an open format and serves as the single source of truth (SSOT) for all the consumption layers. A modern data platform has needs that span traditional data warehouses, data lakes, machine learning systems, and streaming systems and there is some overlap among these systems. A Lakehouse offers features that span all four systems [...]" (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"Many argue that model drift is best monitored by monitoring the data drift in incoming data and the drift in the generated features. As and when the ground truth is available, it is joined by some primary key criteria with the inference data in a Delta table. Again, the update and merge operation support in Delta makes this a breeze. Now the actual and predicted values of the inference data are computed to see how well the model is doing in terms of the quality of insight generation. The feature engineering pipeline is completely in-house and is easier to monitor for drift. The model interpretability may indicate that some columns contributing to the predictive power are incorrect, and it may be necessary to add or remove features. In such cases, a threshold of tolerance is violated, which signals a need for model retraining." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"Metadata is critical in driving business value. It does this by facilitating innovation and collaboration among data teams, which indirectly helps mitigate risks such as misinterpretation and misrepresentation of data. Not only does it help ML practitioners discover the right datasets to use for their modeling exercises, but it also enables citizen data scientists to access the most valuable datasets, thereby ensuring the generation of timely and accurate insights." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"Simply put, 'lakehouse' refers to an open data architecture that combines the best of data lakes and data warehouses on a single platform. At this point, it would be fair to say that a lakehouse is closer to a data lake than a data warehouse. In fact, it is an extension of your data lake to support all use cases, from BI to AI. All data science and ML personas who were shunted into downstream applications because the tools of their trade were so vastly different and can now share the same stage and have access to the same data as other data personas. This eliminates the need to stitch fragile systems together and leads to better data quality and end-to-end latencies since there is no need to copy data across disparate architectures." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"Since data engineering is such a crucial field, you may be wondering who the main players are and what skill sets they possess. Building a data product involves several folks, all of whom need to come together with seamless handoffs to ensure a successful end product or service is created. It would be a mistake to create silos and increase both the number and complexity of integration points as each additional integration is a potential failure point." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"The main challenges include relentlessly chasing data issues that include schema and quality changes (data drift). Sometimes, fixing these issues can cause outages and delays to existing jobs. This is tied tightly to the underlying infrastructure, process, and technology and can be vulnerable to any changes there. For example, a temporary glitch in the cloud ecosystem will result in a failure of the data pipeline." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"Traditional data lakes provide the necessary scalability, but not the real-time concurrency and latency needed for BI use cases. Delta comes to the rescue once again by providing performance at scale with a host of optimization techniques, such as caching, data compaction, and indexing. Previously, a subset of the curated data would be pushed to a warehouse to satisfy the latency and concurrency requirements of known queries. What this meant was that if a consumer needed a different access pattern or a slightly older dataset that was not available, they would have to request that their IT or data team get involved. This took data democratization a step backward. Ideally, we should allow people to access any data that they have privileges to. Delta Lake goes a step forward and allows BI tools to access data directly from the lake instead of accessing a sliver of the data in their expensive warehouses." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"Understanding modern data architectures and sound data engineering principles and practices are crucial to ensure that your AI and BI strategies are reliable and defensible. Generated insights are going to be as good as the quality of the underlying data, so the upfront effort put into understanding the data, modeling it, and transforming it per the business needs goes a long way to foster innovation, productivity, and agility in your data teams." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"We are at the interesting conjunction of big data, the cloud, and artificial intelligence (AI), all of which are fueling tremendous innovation in every conceivable industry vertical and generating data exponentially. Data engineering is increasingly important as data drives business use cases in every industry vertical. You may argue that data scientists and machine learning practitioners are the unicorns of the industry, and they can work their magic for business. That is certainly a stretch of the imagination. Simple algorithms and a lot of good reliable data produce better insights than complicated algorithms with inadequate data." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

🔢Dhanurjay "DJ" Patil - Collected Quotes

"[...] a good definition of a data product is a product that facilitates an end goal through the use of data. It’s tempting to think of a data product purely as a data problem. After all, there’s nothing more fun than throwing a lot of technical expertise and fancy algorithmic work at a difficult problem." (Dhanurjay Patil, "Data Jujitsu: The Art of Turning Data into Product", 2012)

"As data scientists, we prefer to interact with the raw data. We know how to import it, transform it, mash it up with other data sources, and visualize it. Most of your customers can’t do that. One of the biggest challenges of developing a data product is figuring out how to give data back to the user. Giving back too much data in a way that’s overwhelming and paralyzing is 'data vomit'. It’s natural to build the product that you would want, but it’s very easy to overestimate the abilities of your users. The product you want may not be the product they want." (Dhanurjay Patil, "Data Jujitsu: The Art of Turning Data into Product", 2012)

"By giving data back to the user, you can create both engagement and revenue. We’re far enough into the data game that most users have realized that they’re not the customer, they’re the product. Their role in the system is to generate data, either to assist in ad targeting or to be sold to the highest bidder, or both." (Dhanurjay Patil, "Data Jujitsu: The Art of Turning Data into Product", 2012)

"Data Jujitsu: the art of using multiple data elements in clever ways to solve iterative problems that, when combined, solve a data problem that might otherwise be intractable." (Dhanurjay Patil, "Data Jujitsu: The Art of Turning Data into Product", 2012)

"Generalizing beyond advertising, when building any data product in which the data is obfuscated (where there isn’t a clear relationship between the user and the result), you can compromise on precision, but not on recall. But when the data is exposed, focus on high precision." (Dhanurjay Patil, "Data Jujitsu: The Art of Turning Data into Product", 2012)

"Ideas for data products tend to start simple and become complex; if they start complex, they become impossible." (Dhanurjay Patil, "Data Jujitsu: The Art of Turning Data into Product", 2012)

"In many applications, a design treatment that gives the user control over the outcome can go far to create interactions that leave the user feeling good." (Dhanurjay Patil, "Data Jujitsu: The Art of Turning Data into Product", 2012)

"Smart data scientists don’t just solve big, hard problems; they also have an instinct for making big problems small." (Dhanurjay Patil, "Data Jujitsu: The Art of Turning Data into Product", 2012)

"The best way to avoid data vomit is to focus on actionability of data. That is, what action do you want the user to take? If you want them to be impressed with the number of things that you can do with the data, then you’re likely producing data vomit. If you’re able to lead them to a clear set of actions, then you’ve built a product with a clear focus." (Dhanurjay Patil, "Data Jujitsu: The Art of Turning Data into Product", 2012)

"The key aspect of making a data product is putting the 'product' first and 'data' second. Saying it another way, data is one mechanism by which you make the product user-focused. With all products, you should ask yourself the following three questions: (1) What do you want the user to take away from this product? (2) What action do you want the user to take because of the product? (3) How should the user feel during and after using your product?" (Dhanurjay Patil, "Data Jujitsu: The Art of Turning Data into Product", 2012)

"You can give your data product a better chance of success by carefully setting the users’ expectations. [...] One under-appreciated facet of designing data products is how the user feels after using the product. Does he feel good? Empowered? Or disempowered and dejected?" (Dhanurjay Patil, "Data Jujitsu: The Art of Turning Data into Product", 2012)

"Data is such an incredible lever arm for change, we need to make sure that the change that is coming, is the one we all want to see." (Dhanurjay Patil, "A Code of Ethics for Data Science", 2016)

🎯Clay Helberg - Collected Quotes

"Another key element in making informative graphs is to avoid confounding design variation with data variation. This means that changes in the scale of the graphic should always correspond to changes in the data being represented." (Clay Helberg, "Pitfalls of Data Analysis (or How to Avoid Lies and Damned Lies)", 1995) 

"Another trouble spot with graphs is multidimensional variation. This occurs where two-dimensional figures are used to represent one-dimensional values. What often happens is that the size of the graphic is scaled both horizontally and vertically according to the value being graphed. However, this results in the area of the graphic varying with the square of the underlying data, causing the eye to read an exaggerated effect in the graph." (Clay Helberg, "Pitfalls of Data Analysis (or How to Avoid Lies and Damned Lies)", 1995) 

"It may be helpful to consider some aspects of statistical thought which might lead many people to be distrustful of it. First of all, statistics requires the ability to consider things from a probabilistic perspective, employing quantitative technical concepts such as 'confidence', 'reliability', 'significance'. This is in contrast to the way non-mathematicians often cast problems: logical, concrete, often dichotomous conceptualizations are the norm: right or wrong, large or small, this or that." (Clay Helberg, "Pitfalls of Data Analysis (or How to Avoid Lies and Damned Lies)", 1995) 

"[...] many non-mathematicians hold quantitative data in a sort of awe. They have been lead to believe that numbers are, or at least should be, unquestionably correct." (Clay Helberg, "Pitfalls of Data Analysis (or How to Avoid Lies and Damned Lies)", 1995) 

"Most statistical models assume error free measurement, at least of independent (predictor) variables. However, as we all know, measurements are seldom if ever perfect. Particularly when dealing with noisy data such as questionnaire responses or processes which are difficult to measure precisely, we need to pay close attention to the effects of measurement errors. Two characteristics of measurement which are particularly important in psychological measurement are reliability and validity." (Clay Helberg, "Pitfalls of Data Analysis (or How to Avoid Lies and Damned Lies)", 1995) 

"Remember that a p-value merely indicates the probability of a particular set of data being generated by the null model - it has little to say about the size of a deviation from that model (especially in the tails of the distribution, where large changes in effect size cause only small changes in p-values)." (Clay Helberg, "Pitfalls of Data Analysis (or How to Avoid Lies and Damned Lies)", 1995)

"There are a number of ways that statistical techniques can be misapplied to problems in the real world. Three of the most common hazards are designing experiments with insufficient power, ignoring measurement error, and performing multiple comparisons." (Clay Helberg, "Pitfalls of Data Analysis (or How to Avoid Lies and Damned Lies)", 1995)

"We can consider three broad classes of statistical pitfalls. The first involves sources of bias. These are conditions or circumstances which affect the external validity of statistical results. The second category is errors in methodology, which can lead to inaccurate or invalid results. The third class of problems concerns interpretation of results, or how statistical results are applied (or misapplied) to real world issues." (Clay Helberg, "Pitfalls of Data Analysis (or How to Avoid Lies and Damned Lies)", 1995) 

References:
[1] Clay Helberg, "Pitfalls of Data Analysis (or How to Avoid Lies and Damned Lies)", 1995 [link]

⛩️Daniel Jackson - Collected Quotes

"A commitment to simplicity of design means addressing the essence of design - the abstractions on which software is built - explicitly and up front. Abstractions are articulated, explained, reviewed and examined deeply, in isolation from the details of the implementation. This doesn’t imply a waterfall process, in which all design and specification precedes all coding. But developers who have experienced the benefits of this separation of concerns are reluctant to rush to code, because they know that an hour spent on designing abstractions can save days of refactoring." (Daniel Jackson, "Software Abstractions", 2006)

"A language for describing software abstractions is more than just a logic. You need ways to organize a model, to build larger models from smaller ones, and to factor out components that can be used more than once. There are also small syntactic details - such as shorthands for declarations - that make a language usable in practice. And finally, there’s the need to communicate with an analysis tool, by indicating which analyses are to be performed." (Daniel Jackson, "Software Abstractions", 2006)

"A model diagram declares some sets and binary relations, and imposes some basic constraints on them. A diagram is a good way to convey the outline of a model, but diagrams aren’t expressive enough to include detailed constraints." (Daniel Jackson, "Software Abstractions", 2006) 

"Abstractions matter to users too. Novice users want programs whose abstractions are simple and easy to understand; experts want abstractions that are robust and general enough to be combined in new ways. When good abstractions are missing from the design, or erode as the system evolves, the resulting program grows barnacles of complexity. The user is then forced to master a mass of spurious details, to develop workarounds, and to accept frequent, inexplicable failures." (Daniel Jackson, "Software Abstractions", 2006)

"An assertion is a constraint that is intended to follow from the facts of the model. […] Typically, assertions play two different roles. Some express mundane properties that aren’t interesting in their own right; they’re written purely to detect flaws in the model. It’s surprising how effective even a few such assertions can be in uncovering subtle flaws. […] Other assertions express truly essential properties, and are sometimes more fundamental than the facts of the model." (Daniel Jackson, "Software Abstractions", 2006)

"Analysis brings software abstractions to life in three ways. First, it encourages you as you explore, by giving you concrete examples that reinforce intuition and suggest new scenarios. Second, it keeps you honest, by helping you to check as you go along that what you write down means what you think it means. And third, it can reveal subtle fl aws that you might not have discovered until much later (or not at all)." (Daniel Jackson, "Software Abstractions", 2006)

"An abstraction is not a module, or an interface, class, or method; it is a structure, pure and simple - an idea reduced to its essential form. Since the same idea can be reduced to different forms, abstractions are always, in a sense, inventions, even if the ideas they reduce existed before in the world outside the software. The best abstractions, however, capture their underlying ideas so naturally and convincingly that they seem more like discoveries." (Daniel Jackson, "Software Abstractions", 2006)

"Software is built on abstractions. Pick the right ones, and programming will flow naturally from design; modules will have small and simple interfaces; and new functionality will more likely fit in without extensive reorganization […] Pick the wrong ones, and programming will be a series of nasty surprises: interfaces will become baroque and clumsy as they are forced to accommodate unanticipated interactions, and even the simplest of changes will be hard to make." (Daniel Jackson, "Software Abstractions", 2006)

⛩️Alan J Perlis - Collected Quotes

"A language that doesn’t affect the way you think about programming, is not worth knowing." (Alan J Perlis, "Epigrams on Programming", 1982)

"A program without a loop and a structured variable isn’t worth writing." (Alan J Perlis, "Epigrams on Programming", 1982)

"A programming language is low level when its programs require attention to the irrelevant." (Alan J Perlis, "Epigrams on Programming", 1982)

"Adapting old programs to fit new machines usually means adapting new machines to behave like old ones." (Alan J Perlis, "Epigrams on Programming", 1982)

"Computers don’t introduce order anywhere as much as they expose opportunities." (Alan J Perlis, "Epigrams on Programming", 1982)

"Documentation is like term insurance: It satisfies because almost no one who subscribes to it depends on its benefits." (Alan J Perlis, "Epigrams on Programming", 1982)

"Don’t have good ideas if you aren’t willing to be responsible for them." (Alan J Perlis, "Epigrams on Programming", 1982)

"Epigrams retrieve deep semantics from a data base that is all procedure." (Alan J Perlis, "Epigrams on Programming", 1982)

"Every program has (at least) two purposes: the one for which it was written, and another for which it wasn’t." (Alan J Perlis, "Epigrams on Programming", 1982)

"Functions delay binding; data structures induce binding. Moral: Structure data late in the programming process. " (Alan J Perlis, "Epigrams on Programming", 1982)

"If a program manipulates a large amount of data, it does so in a small number of ways." (Alan J Perlis, "Epigrams on Programming", 1982)

"If we believe in data structures, we must believe in independent (hence simultaneous) processing. For why else would we collect items within a structure? Why do we tolerate languages that give us the one without the other?" (Alan J Perlis, "Epigrams on Programming", 1982)

"In programming, everything we do is a special case of something more general — and often we know it too quickly." (Alan J Perlis, "Epigrams on Programming", 1982)

"In seeking the unattainable, simplicity only gets in the way." (Alan J Perlis, "Epigrams on Programming", 1982)

"Interfaces keep things tidy, but don’t accelerate growth: Functions do." (Alan J Perlis, "Epigrams on Programming", 1982)

"It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures." (Alan J Perlis, "Epigrams on Programming", 1982)

"It is easier to change the specification to fit the program than vice versa. " (Alan J Perlis, "Epigrams on Programming", 1982)

"It is easier to write an incorrect program than understand a correct one." (Alan J Perlis, "Epigrams on Programming", 1982)

"It is not a language’s weakness but its strengths that control the gradient of its change: Alas, a language never escapes its embryonic sac." (Alan J Perlis, "Epigrams on Programming", 1982)

"Make no mistake about it: Computers process numbers — not symbols. We measure our understanding (and control) by the extent to which we can arithmetize an activity." (Alan J Perlis, "Epigrams on Programming", 1982)

"Making something variable is easy. Controlling duration of constancy is the trick." (Alan J Perlis, "Epigrams on Programming", 1982)

"Most people find the concept of programming obvious, but the doing impossible." (Alan J Perlis, "Epigrams on Programming", 1982)

"Often it is the means that justify the ends: Goals advance technique and technique survives even when goal structures crumble." (Alan J Perlis, "Epigrams on Programming", 1982)

"One can only display complex information in the mind. Like seeing, movement or flow or alteration of view is more important than the static picture, no matter how lovely." (Alan J Perlis, "Epigrams on Programming", 1982)

"Programmers are not to be measured by their ingenuity and their logic but by the completeness of their case analysis." (Alan J Perlis, "Epigrams on Programming", 1982)

"Prolonged contact with the computer turns mathematicians into clerks and vice versa." (Alan J Perlis, "Epigrams on Programming", 1982)

"Recursion is the root of computation since it trades description for time." (Alan J Perlis, "Epigrams on Programming", 1982)

"Simplicity does not precede complexity, but follows it." (Alan J Perlis, "Epigrams on Programming", 1982)

"Software is under a constant tension. Being symbolic it is arbitrarily perfectible; but also it is arbitrarily changeable." (Alan J Perlis, "Epigrams on Programming", 1982)

"Some programming languages manage to absorb change, but withstand progress." (Alan J Perlis, "Epigrams on Programming", 1982)

"Symmetry is a complexity-reducing concept (co-routines include subroutines); seek it everywhere." (Alan J Perlis, "Epigrams on Programming", 1982)

"Systems have sub-systems and sub-systems have sub-systems and so on ad infinitum - which is why we’re always starting over." (Alan J Perlis, "Epigrams on Programming", 1982)

"The cybernetic exchange between man, computer and algorithm is like a game of musical chairs: The frantic search for balance always leaves one of the three standing ill at ease." (Alan J Perlis, "Epigrams on Programming", 1982)

"The goal of computation is the emulation of our synthetic abilities, not the understanding of our analytic ones." (Alan J Perlis, "Epigrams on Programming", 1982)

"The string is a stark data structure and everywhere it is passed there is much duplication of process. It is a perfect vehicle for hiding information." (Alan J Perlis, "Epigrams on Programming", 1982)

"The use of a program to prove the 4-color theorem will not change mathematics - it merely demonstrates that the theorem, a challenge for a century, is probably not important to mathematics." (Alan J Perlis, "Epigrams on Programming", 1982)

"To understand a program you must become both the machine and the program." (Alan J Perlis, "Epigrams on Programming", 1982)

"We kid ourselves if we think that the ratio of procedure to data in an active data-base system can be made arbitrarily small or even kept small." (Alan J Perlis, "Epigrams on Programming", 1982)

"We will never run out of things to program as long as there is a single program around." (Alan J Perlis, "Epigrams on Programming", 1982)

"Wherever there is modularity there is the potential for misunderstanding: Hiding information implies a need to check communication." (Alan J Perlis, "Epigrams on Programming", 1982)

⛩️Yegor Bugayenko - Collected Quotes

"All companies are built as hierarchies, no matter what that holacracy adepts are saying now. It's always a boss on the top and then people who report to him down to the lowest level. Staying on the lowest level is what I always try to avoid. Not only because I have some dignity, but mostly because I am lazy. The lower you are in the hierarchy, the more work you have to do and the less money you get for it. This is how the division of labor works, not only in the software industry." (Yegor Bugayenko, "Code Ahead", 2018)

"Any software project must have a technical leader, who is responsible for all technical decisions made by the team and have enough authority to make them. Responsibility and authority are two mandatory components that must be present in order to make it possible to call such a person an architect." (Yegor Bugayenko, "Code Ahead", 2018)

"Attributing bugs to their authors doesn't make them more responsible, only more scared." (Yegor Bugayenko, "Code Ahead", 2018)

"Automated testing is a safety net that protects the program from its programmers." (Yegor Bugayenko, "Code Ahead", 2018)

"Every conflict must produce a win-win outcome and must never be resolved through a compromise, which makes both sides suffer in some way. Even forcing one side to do what the other side wants is better than a compromise." (Yegor Bugayenko, "Code Ahead", 2018)

"Fixing the system without fixing people that work in it would be a huge trauma for them; they will do everything they can to prevent it from happening." (Yegor Bugayenko, "Code Ahead", 2018)

"It is not loyalty or internal motivation that drives us programmers forward. We must write our code when the road to our personal success is absolutely clear for us and writing high quality code obviously helps us move forward on this road. To make this happen, the management has to define the rules of the game, also known as "process", and make sure they are strictly enforced, which is much more difficult than 'being agile'." (Yegor Bugayenko, "Code Ahead", 2018)

"It's impossible to change the management system without changing the managers who built it. The management is the product of people who created it." (Yegor Bugayenko, "Code Ahead", 2018)

"Just by making the architect role explicit, a team can effectively resolve many technical conflicts." (Yegor Bugayenko, "Code Ahead", 2018)

"Punishment demotivates when it comes from people rather than a system of well-defined rules." (Yegor Bugayenko, "Code Ahead", 2018)

"Quality is a product of a conflict between programmers and testers." (Yegor Bugayenko, "Code Ahead", 2018)

"Quality must be enforced, otherwise it won't happen. We programmers must be required to write tests, otherwise we won't do it." (Yegor Bugayenko, "Code Ahead", 2018)

"Responsibility means an inevitable punishment for mistakes; authority means full power to make them." (Yegor Bugayenko, "Code Ahead", 2018)

"The higher the price of information in a software team, the less effective the team is." (Yegor Bugayenko, "Code Ahead", 2018)

"The job of a tester is to prove that the software is bug free, while it has to be the other way around: The job of a tester is to prove that the software is broken. The better testers are doing their jobs, the more bugs they manage to find and report." (Yegor Bugayenko, "Code Ahead", 2018)

"To make technical decisions, a result-oriented team needs a strong architect and a decision making process, not meetings." (Yegor Bugayenko, "Code Ahead", 2018)

"Very often managers are just a noise, while the real boss is the project, which we work for and which pays us." (Yegor Bugayenko, "Code Ahead", 2018)

"We must not blame programmers for their bugs. They belong to them only until the code is merged to the repository. After that, all bugs are ours!" (Yegor Bugayenko, "Code Ahead", 2018)

"We, newbies and young programmers, don't like chaos because it makes us dependent on experts. We have to beg for information and feel bad." (Yegor Bugayenko, "Code Ahead", 2018)

⛩️Martin Kleppmann - Collected Quotes

"A fault is usually defined as one component of the system deviating from its spec, where - as a failure is when the system as a whole stops providing the required service to the user. It is impossible to reduce the probability of a fault to zero; therefore it is usually best to design fault-tolerance mechanisms that prevent faults from causing failures." (Martin Kleppmann, "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems", 2015)

"[…] a NoSQL system may find itself accidentally reinventing SQL, albeit in disguise."(Martin Kleppmann, "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems", 2015)

"An architecture that scales well for a particular application is built around assumptions of which operations will be common and which will be rare - the load parameters. If those assumptions turn out to be wrong, the engineering effort for scaling is at best wasted, and at worst counterproductive." (Martin Kleppmann, "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems", 2015)

"[…] as software engineers and architects, we also need to have a technically accurate and precise understanding of the various technologies and their trade-offs if we want to build good applications." (Martin Kleppmann, "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems", 2015)

"[…] building for scale that you don’t need is wasted effort and may lock you into an inflexible design. In effect, it is a form of premature optimization. However, it’s also important to choose the right tool for the job, and different technologies each have their own strengths and weaknesses." (Martin Kleppmann, "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems", 2015)

"Consensus is one of the most important and fundamental problems in distributed computing. On the surface, it seems simple: informally, the goal is simply to get several nodes to agree on something." (Martin Kleppmann, "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems", 2015)

"Every legacy system is unpleasant in its own way, and so it is difficult to give general recommendations for dealing with them." (Martin Kleppmann, "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems", 2015)

"Everybody has an intuitive idea of what it means for something to be reliable or unreliable. For software, typical expectations include: The application performs the function that the user expected. It can tolerate the user making mistakes or using the software in unexpected ways. Its performance is good enough for the required use case, under the expected load and data volume. The system prevents any unauthorized access and abuse." (Martin Kleppmann, "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems", 2015)

"It would be unwise to assume that faults are rare and simply hope for the best. It is important to consider a wide range of possible faults - even fairly unlikely ones - and to artificially create such situations in your testing environment to see what happens. In distributed systems, suspicion, pessimism, and paranoia pay off." (Martin Kleppmann, "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems", 2015)

"Reducing response times at very high percentiles is difficult because they are easily affected by random events outside of your control, and the benefits are diminishing." (Martin Kleppmann, "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems", 2015) 

"Technology is a powerful force in our society. Data, software, and communication can be used for bad: to entrench unfair power structures, to undermine human rights, and to protect vested interests. But they can also be used for good: to make underrepresented people’s voices heard, to create opportunities for everyone, and to avert disasters." (Martin Kleppmann, "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems", 2015)

"The architecture of systems that operate at large scale is usually highly specific to the application - there is no such thing as a generic, one-size-fits-all scalable architecture (informally known as magic scaling sauce)." (Martin Kleppmann, "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems", 2015)

"The fact that SQL is more limited in functionality gives the database much more room for automatic optimizations." (Martin Kleppmann, "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems", 2015)

"The need for data integration often only becomes apparent if you zoom out and consider the dataflows across an entire organization." (Martin Kleppmann, "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems", 2015)

"This is a deliberate choice in the design of computers: if an internal fault occurs, we prefer a computer to crash completely rather than returning a wrong result, because wrong results are difficult and confusing to deal with. Thus, computers hide the fuzzy physical reality on which they are implemented and present an idealized system model that operates with mathematical perfection." (Martin Kleppmann, "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems", 2015)

"[...] when measuring performance, it’s worth using percentiles rather than averages. The main advantage of the mean is that it’s easy to calculate, but percentiles are much more meaningful." (Martin Kleppmann, "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems", 2015)

"When we develop predictive analytics systems, we are not merely automating a human’s decision by using software to specify the rules for when to say yes or no; we are even leaving the rules themselves to be inferred from data. However, the patterns learned by these systems are opaque: even if there is some correlation in the data, we may not know why. If there is a systematic bias in the input to an algorithm, the system will most likely learn and amplify that bias in its output." (Martin Kleppmann, "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems", 2015)

"Working with distributed systems is fundamentally different from writing software on a single computer - and the main difference is that there are lots of new and exciting ways for things to go wrong." (Martin Kleppmann, "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems", 2015)

⛩️Orson Scott Card - Collected Quotes

"The computer is also not famous for having mercy." (Orson Scott Card, "Ender’s Game", 1985)

"Coincidence is just the word we use when we have not yet discovered the cause." (Orson Scott Card, "The Call Of Earth", 1992)

"Here's the secret that every successful software company is based on: You can domesticate programmers the way beekeepers tame bees. You can't exactly communicate with them, but you can get them to swarm in one place and when they're not looking, you can carry off the honey." (Orson Scott Card, "How Software Companies Die", Windows Sources: The Magazine for Windows Experts, 1995)

"Programming is the Great Game. It consumes you, body and soul. When you're caught up in it, nothing else matters. When you emerge into daylight, you might well discover that you're a hundred pounds overweight, your underwear is older than the average first grader, and judging from the number of pizza boxes lying around, it must be spring already. But you don't care, because your program runs, and the code is fast and clever and tight. You won." (Orson Scott Card, "How Software Companies Die", Windows Sources: The Magazine for Windows Experts, 1995)

"The environment that nuture's creative programmers kills management and marketing types - and vice versa." (Orson Scott Card, "How Software Companies Die", Windows Sources: The Magazine for Windows Experts, 1995)

"We don’t use the word ‘intelligence’ with software. We regard that as a naive idea. We say that it’s ‘complex.’ Which means that we don’t always understand what it’s doing." (Orson Scott Card, "Ender's Shadow", 1999)

"Leading is a strange thing […]. People see it happening, but they don’t have a clue how it works. […] They don’t see what a leader does, they just see how everybody respects a good leader, and they want to have the attention and respect without understanding what you actually have to do to earn it." (Orson Scott Card, "Ender in Exile", 2008)

"Some people are born to lead [...].They just think that way, whether they want to lead or not. While others are born craving authority, but they have no ability to lead." (Orson Scott Card, "Ender in Exile", 2008)

"The real training ground for leadership is in the game." (Orson Scott Card, "First Meetings in Ender's Universe", 2002)

"The leader only has as much power as his followers give him." (Orson Scott Card, "Ender in Exile", 2008)

"We arrive at an extremely high level of technology - but with nothing under it to hold it up. If we crash, we crash all the way down." (Orson Scott Card, "Ender in Exile", 2008)

"You can’t lead people you don’t know or at least understand." (Orson Scott Card, "Ender in Exile", 2008)

"Computers were a kind of magery in themselves, or might as well be - to people who didn't understand them, they were every bit as inscrutable." (Orson Scott Card, The Lost Gate", 2010)

"Humans make a machine, and then fool themselves into believing that their own brains are no better than the machines. This allows them to believe that their creation, the computer, is as brilliant as their own minds. But it’s a ridiculous self-deception. Computers aren’t even in the same league." (Orson Scott Card, "Ruins", 2013)

⛩️Titus Winters - Collected Quotes

"A boat without a captain is nothing more than a floating waiting room: unless someone grabs the rudder and starts the engine, it’s just going to drift along aimlessly with the current. A piece of software is just like that boat: if no one pilots it, you’re left with a group of engineers burning up valuable time, just sitting around waiting for something to happen (or worse, still writing code that you don’t need)." (Titus Winters, "Software Engineering at Google: Lessons Learned from Programming Over Time", 2020)

"Code coverage can provide some insight into untested code, but it is not a substitute for thinking critically about how well your system is tested." (Titus Winters, "Software Engineering at Google: Lessons Learned from Programming Over Time", 2020)

"Fixing a bug is much like adding a new feature: the presence of the bug suggests that a case was missing from the initial test suite, and the bug fix should include that missing test case." (Titus Winters, "Software Engineering at Google: Lessons Learned from Programming Over Time", 2020)

"[...] do not underestimate the power of playing the social game. It’s not about tricking or manipulating people; it’s about creating relationships to get things done. Relationships always outlast projects. When you’ve got richer relationships with your coworkers, they’ll be more willing to go the extra mile when you need them." (Titus Winters, "Software Engineering at Google: Lessons Learned from Programming Over Time", 2020)

"Given enough time and enough users, even the most innocuous change will break something; your analysis of the value of that change must incorporate the difficulty in investigating, identifying, and resolving those breakages." (Titus Winters, "Software Engineering at Google: Lessons Learned from Programming Over Time", 2020)

"In a professional software engineering environment, criticism is almost never personal - it’s usually just part of the process of making a better project. The trick is to make sure you (and those around you) understand the difference between a constructive criticism of someone’s creative output and a flat-out assault against someone’s character." (Titus Winters, "Software Engineering at Google: Lessons Learned from Programming Over Time", 2020)

"In addition to developing the proper culture, invest in your testing infrastructure by developing linters, documentation, or other assistance that makes it more difficult to write bad tests." (Titus Winters, "Software Engineering at Google: Lessons Learned from Programming Over Time", 2020)

"In short, performance ratings are indicative only of how a person is performing in their given role at the time they are being evaluated. Ratings, although an important way to measure performance during a specific period, are not predictive of future performance and should not be used to gauge readiness for a future role or qualify an internal candidate for a different team. (They can, however, be used to evaluate whether an employee is properly or improperly slotted on their current team; therefore, they can provide an opportunity to evaluate how to better support an internal candidate moving forward.)" (Titus Winters, "Software Engineering at Google: Lessons Learned from Programming Over Time", 2020)

"Knowledge is in some ways the most important (though intangible) capital of a software engineering organization, and sharing of that knowledge is crucial for making an organization resilient and redundant in the face of change. A culture that promotes open and honest knowledge sharing distributes that knowledge efficiently across the organization and allows that organization to scale over time. In most cases, investments into easier knowledge sharing reap manyfold dividends over the life of a company." (Titus Winters, "Software Engineering at Google: Lessons Learned from Programming Over Time", 2020)

"Making good engineering decisions is all about weighing all of the available inputs and making informed decisions about the trade-offs. Sometimes, those decisions are based on instinct or accepted best practice, but only after we have exhausted approaches that try to measure or estimate the true underlying costs." (Titus Winters, "Software Engineering at Google: Lessons Learned from Programming Over Time", 2020)

"One of the broad truths we’ve seen to be true is the idea that finding problems earlier in the developer workflow usually reduces costs." (Titus Winters, "Software Engineering at Google: Lessons Learned from Programming Over Time", 2020)

"People are inherently imperfect - we like to say that humans are mostly a collection of intermittent bugs. But before you can understand the bugs in your coworkers, you need to understand the bugs in yourself. We’re going to ask you to think about your own reactions, behaviors, and attitudes - and in return, we hope you gain some real insight into how to become a more efficient and successful software engineer who spends less energy dealing with people-related problems and more time writing great code." (Titus Winters, "Software Engineering at Google: Lessons Learned from Programming Over Time", 2020)

"Programming is the immediate act of producing code. Software engineering is the set of policies, practices, and tools that are necessary to make that code useful for as long as it needs to be used and allowing collaboration across a team." (Titus Winters, "Software Engineering at Google: Lessons Learned from Programming Over Time", 2020)

"This is what good management is about: 95% observation and listening, and 5% making critical adjustments in just the right place." (Titus Winters, "Software Engineering at Google: Lessons Learned from Programming Over Time", 2020)

"When an engineer refactors the internals of a system without modifying its interface, whether for performance, clarity, or any other reason, the system’s tests shouldn’t need to change. The role of tests in this case is to ensure that the refactoring didn’t change the system’s behavior. Tests that need to be changed during a refactoring indicate that either the change is affecting the system’s behavior and isn’t a pure refactoring, or that the tests were not written at an appropriate level of abstraction." (Titus Winters, "Software Engineering at Google: Lessons Learned from Programming Over Time", 2020)

⛩️Eddie Burris - Collected Quotes

"A design pattern, and more generally a design, is an abstraction of an implementation. Because the human mind is better at forming abstractions from details, maybe the best way to get a deeper understanding of design patterns is to reverse engineer one from implementation." (Eddie Burris, "Programming in the Large with Design Patterns", 2012)

"A software design pattern is a reusable solution to a reoccurring software design problem. […] A design pattern is not a concrete design or implementation, such as an algorithm that can be used as-is, but rather a gene." (Eddie Burris, "Programming in the Large with Design Patterns", 2012)

"Algorithm selection is often a design time decision but there are times when it is necessary or desirable to postpone algorithm selection until runtime. Runtime selection is preferable when the choice of algorithm depends on factors not available at design time. Such factors include: nature of input, source of input, user preferences and current conditions." (Eddie Burris, "Programming in the Large with Design Patterns", 2012)

"Design patterns make design easier but not easy. Their application still requires a modest amount of reasoning and problem solving." (Eddie Burris, "Programming in the Large with Design Patterns", 2012)

"Knowledge of design patterns simplifies software design by reducing the number of design problems that have to be solved from first principles. Design problems that match documented design patterns have ready-made solutions. The remaining. problems that don't match documented design patterns must be solved from first principles. Even here, knowledge of design patterns can potentially help with original design. Design patterns are paragons of good design. Studying design patterns helps to develop the intellectual concepts and principles needed to solve unique design problems from first principles." (Eddie Burris, "Programming in the Large with Design Patterns", 2012)

"Not all problem-solution pairs are patterns. If the concept of a pattern is applied too loosely, it could dilute what it means to be a pattern. Although there is no formal criteria or litmus test for what is and isn’t a pattern, there are a few problem-solution pairs that are not generally thought of as patterns." (Eddie Burris, "Programming in the Large with Design Patterns", 2012)

"One of the benefits of knowing certain design patterns is that it makes it easier to learn class libraries based on these patterns." (Eddie Burris, "Programming in the Large with Design Patterns", 2012)

"The most common form of reuse in software development is code reuse. Libraries of reusable components (routines, classes, packages) are made available to programmers for integration (linking) into their applications. Design patterns enable another form of reuse - design reuse. Practicing design reuse means looking for a routine design before resorting to the creation of an original design." (Eddie Burris, "Programming in the Large with Design Patterns", 2012)

"What makes a design pattern unique is its intent. The intent of a pattern is the problem solved or reason for using it. […] What distinguishes one pattern from another is the problem solved. You can infer the solution structure and problem solved from the pattern name but you can’t infer the pattern name from the solution structure alone." (Eddie Burris, "Programming in the Large with Design Patterns", 2012) 

⛩️Mark W Maier - Collected Quotes

"A systems approach is one that focuses on the system as a whole, specifically linking value judgments (what is desired) and design decisions (what is feasible). A true systems approach means that the design process includes the 'problem' as well as the solution. The architect seeks a joint problem–solution pair and understands that the problem statement is not fixed when the architectural process starts. At the most fundamental level, systems are collections of different things that together produce results unachievable by the elements alone."  (Mark W Maier, "The Art Systems of Architecting" 3rd Ed., 2009)

"Architecting is both an art and a science - both synthesis and analysis, induction and deduction, and conceptualization and certification - using guidelines from its art and methods from its science. As a process, it is distinguished from systems engineering in its greater use of heuristic reasoning, lesser use of analytics, closer ties to the client, and particular concern with certification of readiness for use."  (Mark W Maier, "The Art Systems of Architecting" 3rd Ed., 2009)

"Architecting is creating and building structures - that is, 'structuring'. Systems architecting is creating and building systems. It strives for fit, balance, and compromise among the tensions of client needs and resources, technology, and multiple stakeholder interests."  (Mark W Maier, "The Art Systems of Architecting" 3rd Ed., 2009)

"Insight, or the ability to structure a complex situation in a way that greatly increases understanding of it, is strongly guided by lessons learned from one’s own or others’ experiences and observations. Given enough lessons, their meaning can be codified into succinct expressions called 'heuristics', a Greek term for guide. Heuristics are an essential complement to analytics, particularly in situations where analysis alone cannot provide either insights or guidelines."  (Mark W Maier, "The Art Systems of Architecting" 3rd Ed., 2009)

"It is generally agreed that increasing complexity is at the heart of the most difficult problems facing today’s systems architecting and engineering. When architects and builders are asked to explain cost overruns and schedule delays, by far the most common, and quite valid, explanation is that the system is much more complex than originally thought. The greater is the complexity, the greater the difficulty. It is important, therefore, to understand what is meant by system complexity if architectural progress is to be made in dealing with it."  (Mark W Maier, "The Art Systems of Architecting" 3rd Ed., 2009)

"Less apparent is that qualitatively different problem-solving techniques are required at high levels of complexity than at low ones. Purely analytical techniques, powerful for the lower levels, can be overwhelmed at the higher ones. At higher levels, architecting methods, experience-based heuristics, abstraction, and integrated modeling must be called into play."  (Mark W Maier, "The Art Systems of Architecting" 3rd Ed., 2009)

"Modeling is the creation of abstractions or representations of the system to predict and analyze performance, costs, schedules, and risks and to provide guidelines for systems research, development, design, manufacture, and management. Modeling is the centerpiece of systems architecting - a mechanism of communication to clients and builders, of design management with engineers and designers, of maintaining system integrity with project management, and of learning for the architect, personally."  (Mark W Maier, "The Art Systems of Architecting" 3rd Ed., 2009)

"Modeling is the fabric of architecting because architecting is at a considerable distance of abstraction from actual construction. The architect does not manipulate the actual elements of construction. The architect builds models that are passed into more detailed design processes. Those processes lead, eventually, to construction drawings or the equivalent and actual system fabrication or coding."  (Mark W Maier, "The Art Systems of Architecting" 3rd Ed., 2009)

"Taking a systems approach means paying close attention to results, the reasons we build a system. Architecture must be grounded in the client’s/user’s/customer’s purpose. Architecture is not just about the structure of components. One of the essential distinguishing features of architectural design versus other sorts of engineering design is the degree to which architectural design embraces results from the perspective of the client/user/customer. The architect does not assume some particular problem formulation, as “requirements” is fixed. The architect engages in joint exploration, ideally directly with the client/user/customer, of what system attributes will yield results worth paying for."  (Mark W Maier, "The Art Systems of Architecting" 3rd Ed., 2009)

"The basic idea behind all of these techniques is to simplify problem solving by concentrating on its essentials. Consolidate and simplify the objectives. Focus on the things with the highest impact, things that determine other things. Put to one side minor issues likely to be resolved by the resolution of major ones. Discard the nonessentials. Model (abstract) the system at as high a level as possible, then progressively reduce the level of abstraction. In short: Simplify!"  (Mark W Maier, "The Art Systems of Architecting" 3rd Ed., 2009)

"This primacy of complexity in system design helps explain why a single 'optimum' seldom if ever exists for such systems. There are just too many variables. There are too many stakeholders and too many conflicting interests. No practical way may exist for obtaining information critical in making a 'best' choice among quite different alternatives."  (Mark W Maier, "The Art Systems of Architecting" 3rd Ed., 2009)