SQL Reloaded: Misusing Views and Pseudo-Constants

   Views as virtual tables can be misused to replace tables in certain circumstances, either by storing values within one or multiple rows, like in the below examples:

-- parameters for a BI solution
CREATE VIEW dbo.vLoV_Parameters
AS
SELECT Cast('ABC' as nvarchar(20)) AS DataAreaId
 , Cast(GetDate() as Date) AS CurrentDate 
 , Cast(100 as int) AS BatchCount 

GO

SELECT *
FROM dbo.vLoV_Parameters

GO

-- values for a dropdown 
 CREATE VIEW dbo.vLoV_DataAreas
 AS
 SELECT Cast('ABC' as nvarchar(20)) AS DataAreaId
 , Cast('Company ABC' as nvarchar(50)) AS Description 
 UNION ALL
 SELECT 'XYZ' DataAreaId 
 , 'Company XYZ'

GO

SELECT *
FROM dbo.vLoV_DataAreas

GO

    These solutions aren’t elegant, and typically not recommended because they go against one of the principles of good database design, namely “data belong in tables”, though they do the trick when needed. Personally, I used them only in a handful of cases, e.g. when it wasn’t allowed to create tables, when it was needed testing something for a short period of time, or when there was some overhead of creating a table for 2-3 values. Because of their scarce use, I haven’t given them too much thought, not until I discovered Jared Ko’s blog posting on pseudo-constants. He considers the values from the first view as pseudo-constants, and advocates for their use especially for easier dependency tracking, easier code refactoring, avoiding implicit data conversion and easier maintenance of values.

   All these are good reasons to consider them, therefore I tried to take further the idea to see if it survives a reality check. For this I took Dynamics AX as testing environment, as it makes extensive use of enumerations (aka base enums) to store list of values needed allover through the application. Behind each table there are one or more enumerations, the tables storing master data abounding of them.  For exemplification let’s consider InventTrans, table that stores the inventory transactions, the logic that governs the receipt and issued transactions are governed by three enumerations: StatusIssue, StatusReceipt and Direction.

-- Status Issue Enumeration 
 CREATE VIEW dbo.vLoV_StatusIssue
 AS
 SELECT cast(0 as int) AS None
 , cast(1 as int) AS Sold
 , cast(2 as int) AS Deducted
 , cast(3 as int) AS Picked
 , cast(4 as int) AS ReservPhysical
 , cast(5 as int) AS ReservOrdered
 , cast(6 as int) AS OnOrder
 , cast(7 as int) AS QuotationIssue

GO

-- Status Receipt Enumeration 
 CREATE VIEW dbo.vLoV_StatusReceipt
 AS
SELECT cast(0 as int) AS None
 , cast(1 as int) AS Purchased
 , cast(2 as int) AS Received
 , cast(3 as int) AS Registered
 , cast(4 as int) AS Arrived
 , cast(5 as int) AS Ordered
 , cast(6 as int) AS QuotationReceipt

GO

-- Inventory Direction Enumeration 
 CREATE VIEW dbo.vLoV_InventDirection
 AS
 SELECT cast(0 as int) AS None
 , cast(1 as int) AS Receipt
 , cast(2 as int) AS Issue

   To see these views at work let’s construct the InventTrans table on the fly:

-- creating an ad-hoc table  
 SELECT *
 INTO  dbo.InventTrans
 FROM (VALUES (1, 1, 0, 2, -1, 'A0001')
 , (2, 1, 0, 2, -10, 'A0002')
 , (3, 2, 0, 2, -6, 'A0001')
 , (4, 2, 0, 2, -3, 'A0002')
 , (5, 3, 0, 2, -2, 'A0001')
 , (6, 1, 0, 1, 1, 'A0001')
 , (7, 0, 1, 1, 50, 'A0001')
 , (8, 0, 2, 1, 100, 'A0002')
 , (9, 0, 3, 1, 30, 'A0003')
 , (10, 0, 3, 1, 20, 'A0004')
 , (11, 0, 1, 2, 10, 'A0001')
 ) A(TransId, StatusIssue, StatusReceipt, Direction, Qty, ItemId)

    Here are two sets of examples using literals vs. pseudo-constants:

--example issued with literals 
SELECT top 100 ITR.*
 FROM dbo.InventTrans ITR
 WHERE ITR.StatusIssue = 1 
   AND ITR.Direction = 2

GO
 --example issued with pseudo-constants
 SELECT top 100 ITR.*
 FROM dbo.InventTrans ITR
      JOIN dbo.vLoV_StatusIssue SI
        ON ITR.StatusIssue = SI.Sold
      JOIN dbo.vLoV_InventDirection ID
        ON ITR.Direction = ID.Issue

GO

--example receipt with literals 
 SELECT top 100 ITR.*
 FROM dbo.InventTrans ITR
 WHERE ITR.StatusReceipt= 1
   AND ITR.Direction = 1

GO

--example receipt with pseudo-constants
 SELECT top 100 ITR.*
 FROM dbo.InventTrans ITR
      JOIN dbo.vLoV_StatusReceipt SR
        ON ITR.StatusReceipt= SR.Purchased
      JOIN dbo.vLoV_InventDirection ID
        ON ITR.Direction = ID.Receipt

 
  As can be seen the queries using pseudo-constants make the code somehow readable, though the gain is only relative, each enumeration implying an additional join. In addition, when further business tables are added to the logic (e.g. items, purchases or sales orders)  it complicates the logic, making it more difficult to separate the essential from nonessential. Imagine a translation of the following query:

-- complex query 
  SELECT top 100 ITR.*
  FROM dbo.InventTrans ITR
              <several tables here>
  WHERE ((ITR.StatusReceipt<=3 AND ITR.Direction = 1)
    OR (ITR.StatusIssue<=3 AND ITR.Direction = 2))
    AND (<more constraints here>)

   The more difficult the constraints in the WHERE clause, the more improbable is a translation of the literals into pseudo-constraints. Considering that an average query contains 5-10 tables, each of them with 1-3 enumerations, the queries would become impracticable by using pseudo-constants and quite difficult to troubleshoot their execution plans.

    The more I’m thinking about, an enumeration data type as global variable in SQL Server (like the ones available in VB) would be more than welcome, especially because values are used over and over again through the queries. Imagine, for example, the possibility of writing code as follows:

-- hypothetical query
SELECT top 100 ITR.*
FROM dbo.InventTrans ITR
WHERE ITR.StatusReceipt = @@StatusReceipt .Purchased
  AND ITR.Direction = @@InventDirection.Receipt

   From my point of view this would make the code more readable and easier to maintain. Instead, in order to make the code more readable, one’s usually forced to add some comments in the code. This works as well, though the code can become full of comments.

-- query with commented literals
SELECT top 100 ITR.*
FROM dbo.InventTrans ITR
WHERE ITR.StatusReceipt <=3 – Purchased, Received, Registered 
   AND ITR.Direction = 1-- Receipt

   In conclusion, pseudo-constants’ usefulness is only limited, and their usage is  against developers’ common sense, however a data type in SQL Server with similar functionality would make code more readable and easier to maintain.

PS: It is possible to simulate an enumeration data type in tables’ definition by using a CHECK constraint.

Data Warehousing: ETL (SSIS packages vs. SQL code)

I’ve been working with DTS (Data Transformation Services) packages for the past 7-8 years, finding SQL Server’s 2000 functionality pretty useful in handling ETL (Extract, Transform, Load) tasks, especially when importing and exporting data between multiple data sources. The functionality provided by the DTS platform was basic, though it could be extended in custom packages with code developed in VB or by using ActiveX tasks in VBScript/Jscript, when the SQL-based logic was not enough. In addition, all three types of coding could make calls to .dlls, and thus operating system, vendor or in-house built libraries or simple functions could be (re)used. 

Starting with SQL Server 2005, DTS was replaced by the SSIS (SQL Server Integration Services) component, becoming with SSRS (SQL Reporting Services) and SSAS (SQL Server Analysis Services) integrant part of Microsoft's data platform. Besides providing a new architecture, SSIS extended the functionality previously available, bringing more flexibility in constructing the packages, their elements and data manipulation. SSIS became with each new version of SQL Server (2008, 2012, 2014, 2016, 2017, 2019) a powerful ETL tool that can be easily used for Data Warehousing, Data Migration or Data Integration projects. 

In what concerns ETL we could say there are two main philosophies - have as much of the business logic in the (ETL) package, or use the package mainly for loading data from the various sources and have the business logic in the database as SQL-based code. As always, each of the two philosophies has its own advantages and disadvantages, though I would consider a third philosophy – design for performance, reuse and maintainability, resulting thus a hybrid between the first two mentioned philosophies. 

There are several other factors that need to be considered when building ETL solutions - synchronization, testability, security, stability, scalability, complexity, learning curve, etc. I would say that there is no perfect receipe, no architecture matching all requirements, each solution coming with its needs and constraints, sometimes being a good idea to go one level of abstraction above the requirements, while other times is better to stick to the requirements and problem at hand. 

Performance

Designing for performance resumes in choosing the architecture/methods that provides the best performance individually and as a whole – either using package-based functionality, SQL-based functionality or a combination of both. In general SQL code is best suited for query-based data manipulation, while packages are better suited for sequential or workflow-based processing of data, though there could be exceptions and exceptions. Often it’s a good idea to test the performance of all alternative approaches via a prototype, even if in time the developer arrives to a good knowledge of the methods that suit best from a performance point of view.

Reuse

Each of the two architectures allow a lower or higher degree of reuse using parameters, variables and compartmenting of code, maybe with a plus for SQL code which has in theory a greater maturity and flexibility than the package-based functionality, allowing a wider range of reuse resulted from the compartmenting of code in the various supported objects (stored procedures, functions, views or tables).

Maintainability

Maintainability, the easiness of modifying packages or code, is an important factor because few are the cases in which the logic doesn’t change over time, many projects having to deal with change in requirements, sometimes implying a 180 degrees change of the overall perspective. Therefore it's important to be able to modify the code/package and even change the architecture with a minimum of effort.

Refactoring

Refactoring resumes in modifying the code without changing the functional behavior in order to improve the performance, readability, maintainability or extensibility, minimize the use of resources, remove redundant code, adhere to standards, best practices or naming conventions. It is said that there is always place for improvement, performance being the dimension with the most important impact in the world of databases. Refactoring is not always necessary, though it’s a good practice for achieving high quality solutions.

Synchronization

I would say there are three types of synchronization – of scope, business logic and data, the first targeting to synchronize the filters used to select the data, the second to synchronize the logic used in data processing, when is the case, and third to work with the same unaltered copy of the data. Considering the example of Invoices – Headers and Lines, the synchronization of scope would resume to apply the same constraints for the two, assuring that there is no Invoice Header without Lines, and vice-versa; the data synchronization between the two referring to the fact that the data between the two data sets should be consistent, there should be no change in Invoice Headers not reflected also at Line level (e.g. total amount matching between Lines and Header). Business logic synchronization refers typically to the use of the same set of data for similar purposes, if several transformations were used for Invoice Headers, they should be reflected accordingly also at Invoice Line level. Synchronization it’s actually quite an important topic, therefore I will reconsider it in a further post.

Testing & Troubleshooting

I find that the business logic implemented in SQL code is much easier to test than the logic implemented in packages, because in the first situation each object and step could be in theory tested individually/progressively, being thus much easier to troubleshoot. Troubleshooting packages logic can be quite complex because is not always possible to view the input/output for each intermediary step.

Complexity

Complexity is reflected in the easiness of understanding the logic broken down to pieces and as a whole. Packages are highly visual, being in theory easier to identify and understand the steps and their flow, even by non-technical people, while SQL-code might need auxiliary representations for the same purpose (e.g. data flow diagram) and need a higher level of expertize. 

Security

Security is always a sensitive and complex topic, and in general it resumes to how secure is the code and sensitive information stored (e.g. user name & password, data), who has access to execute it and the context in which the code is run. This can become easily quite a complex topic, being highly dependent on the architecture used.

Stability

We can discuss about platform and design stability, which can be often a matter of perception and experience. Both SSIS and database engines could be considered as stable development environments, the later having in theory a greater maturity and flexibility, flexibility that could be easily brought to extreme creating bad coding monsters (e.g. loop calls to .dll libraries), impacting thus design’s stability, which is correlated to the adequate use of functionality, techniques and resources – each technology has its does and don’ts, strength and weakness.

Deployment
 
Deployment of business logic on the server can be quite easy or quite complex, depending on the overall architecture, the number of configurable items in scope, and the complexity of the dependencies existing between them. The deployment usually resumes at copying the code from one location to another, installing the eventual dependencies and configuring the objects for use.

Scalability

Scalability in this context refers mainly to the degree the business logic can cope with the increased volume of records, and not necessarily with the number of requests, though this aspect could be considered too, after case. SSIS and the database engines are designed to be highly scalable, though there are architectures and architectures, good uses and misuses of techniques. Designing for performance in theory equates with good scalability, unless the requirements makes it difficult to have a scalable solution.

Learning curve

The learning curve of technologies is always an important factor that needs to be considered in development, as it reflects how much time an average developer needs to master the basic/average/complex functionality provided by the respective technology. For ETL development is in general requested an average knowledge of both - SSIS architecture, respectively SQL-based programming - though it’s not easy to acquire both problem-solving mindsets. A SSIS developer usually attempts using as much as possible the functionality provided by SSIS, while a SQL developer the SQL-based functionality. In the end, it’s important to know how to balance between the two.

Previous Post <<||>> Next Post

Software Engineering: Refactoring (Just the Quotes)

"When you feel the need to write a comment, first try to refactor the code so that any comment becomes superfluous." (Kent Beck, "Refactoring: Improving the Design of Existing Code", 1999)

"When you find you have to add a feature to a program, and the program's code is not structured in a convenient way to add the feature, first refactor the program to make it easy to add the feature, then add the feature."  (Kent Beck, "Refactoring: Improving the Design of Existing Code", 1999)

"Unit tests can be tedious to write, but they save you time in the future (by catching bugs after changes). Less obviously, but just as important, is that they can save you time now: tests focus your design and implementation on simplicity, they support refactoring, and they validate features as you develop." (Ron Jeffries, "Extreme Programming Installed, 2001)

"Before you start refactoring, check that you have a solid suite of tests. These tests must be self-checking." (Martin Fowler et al, "Refactoring: Improving the Design of Existing Code", 2002)

"Given software engineers’ infatuation with indirection, it may not surprise you to learn that most refactoring introduces more indirection into a program. Refactoring tends to break big objects into several smaller ones and big methods into several smaller ones." (Kent Beck, "Indirection and Refactoring", 2002)

"One problem area for refactoring is databases. Most business applications are tightly coupled to the database schema that supports them. That's one reason that the database is difficult to change. Another reason is data migration. Even if you have carefully layered your system to minimize the dependencies between the database schema and the object model, changing the database schema forces you to migrate the data, which can be a long and fraught task." (Martin Fowler et al, "Refactoring: Improving the Design of Existing Code", 2002)

"Refactoring is risky. It requires changes to working code that can introduce subtle bugs. Refactoring, if not done properly, can set you back days, even weeks. And refactoring becomes riskier when practiced informally or ad hoc." (Erich Gamma, 2002)

"Refactoring is the process of changing a software system in such a way that it does not alter the external behavior of the code yet improves its internal structure. It is a disciplined way to clean up code that minimizes the chances of introducing bugs. In essence when you refactor you are improving the design of the code after it has been written." (Martin Fowler et al, "Refactoring: Improving the Design of Existing Code", 2002)

"Refactoring is the process of taking a running program and adding to its value, not by changing its behavior but by giving it more of these qualities that enable us to continue developing at speed." (Kent Beck, "Why Refactoring Works", 2002)

"Refactoring changes the programs in small steps. If you make a mistake, it is easy to find the bug." (Martin Fowler et al, "Refactoring: Improving the Design of Existing Code", 2002)

"Without refactoring, the design of the program will decay. As people change code - changes to realize short-term goals or changes made without a full comprehension of the design of the code - the code loses its structure. It becomes harder to see the design by reading the code. Refactoring is rather like tidying up the code. Work is done to remove bits that aren't really in the right place. Loss of the structure of code has a cumulative effect. The harder it is to see the design in the code, the harder it is to preserve it, and the more rapidly it decays. Regular refactoring helps code retain its shape." (Martin Fowler et al, "Refactoring: Improving the Design of Existing Code", 2002)

"A commitment to simplicity of design means addressing the essence of design - the abstractions on which software is built - explicitly and up front. Abstractions are articulated, explained, reviewed and examined deeply, in isolation from the details of the implementation. This doesn’t imply a waterfall process, in which all design and specification precedes all coding. But developers who have experienced the benefits of this separation of concerns are reluctant to rush to code, because they know that an hour spent on designing abstractions can save days of refactoring." (Daniel Jackson, "Software Abstractions", 2006)

"To programmers, refactoring means rewriting a chunk of code to make it briefer, clearer, and easier to read without changing what it actually does. Refactoring is often compared to gardening; it is never finished." (Scott Rosenberg, "Dreaming in Code", 2007)

"It is a myth that we can get systems 'right the first time'. Instead, we should implement only today’s stories, then refactor and expand the system to implement new stories tomorrow. This is the essence of iterative and incremental agility. Test-driven development, refactoring, and the clean code they produce make this work at the code level." (Robert C Martin, "Clean Code: A Handbook of Agile Software Craftsmanship", 2008)

"Refactoring is a lot like solving a Rubik’s cube. There are lots of little steps required to achieve a large goal. Each step enables the next." (Robert C Martin, "Clean Code: A Handbook of Agile Software Craftsmanship", 2008)

"Refactoring is the process of improving the design of existing code without changing its behavior. [...] Bug fixing often uncovers opportunities for refactoring. The very fact that you’re working with code that contains a bug indicates that there is a chance that it could be clearer or better structured." (Paul Butcher, "Debug It! Find, Repair, and Prevent Bugs in Your Code", 2009)

"Although it is focused on the code, refactoring has a large impact on the design of a system. It is vital for senior designers and architects to understand the principles of refactoring and to use them in their projects." (Jay Fields et al, "Refactoring: Ruby Edition", 2010)

"Refactoring is the process of changing a software system in such a way that it does not alter the external behavior of the code yet improves its internal structure. It is a disciplined way to clean up code that minimizes the chances of introducing bugs. In essence when you refactor you are improving the design of the code after it has been written." (Jay Fields et al, "Refactoring: Ruby Edition", 2010)

"When an engineer refactors the internals of a system without modifying its interface, whether for performance, clarity, or any other reason, the system’s tests shouldn’t need to change. The role of tests in this case is to ensure that the refactoring didn’t change the system’s behavior. Tests that need to be changed during a refactoring indicate that either the change is affecting the system’s behavior and isn’t a pure refactoring, or that the tests were not written at an appropriate level of abstraction." (Titus Winters, "Software Engineering at Google: Lessons Learned from Programming Over Time", 2020)

Software Engineering: Refactoring (Definitions)

"Rebuilding and restructuring code in very small steps without changing the behavior visible from outside." (Johannes Link & Peter Fröhlich, "Unit Testing in Java", 2003)

"Revising software design and code to improve their structure, without changing the functions and behavior of the software, in order to facilitate future modification and enhancement of the software. Used in many agile methods. Refactoring is a broader term than code restructuring." (Richard D Stutzke, "Estimating Software-Intensive Systems: Projects, Products, and Processes", 2005)

"A programming technique that enables you to evolve your code slowly over time, to take an evolutionary approach to programming." (Pramod J Sadalage & Scott W Ambler, "Refactoring Databases: Evolutionary Database Design", 2006)

"A simple change to source code that retains its behavioral semantics: You neither add functionality when you are refactoring nor take it away. A refactoring merely improves the design of your code - nothing more and nothing less." (Pramod J Sadalage & Scott W Ambler, "Refactoring Databases: Evolutionary Database Design", 2006)

"Refactoring is used to restructure existing software regarding aspects like modularity, flexibility, and extensibility, without changing the main software functionality." (Alke Martens & Andreas Harrer, "Software Engineering in e-Learning Systems", Encyclopedia of Information Technology Curriculum Integration, 2008)

"To clarify and simplify computer code." (Michael Coles & Rodney Landrum, , "Expert SQL Server 2008 Encryption", 2008)

"A structural transformation that provides a systematic way of eradicating the undesirable(s) from an artifact while preserving its behavioral semantics." (Pankaj Kamthan, "Pattern-Oriented Use Case Modeling", 2009)

[database refactoring:] "It indicates little changes in the database schema which preserve both the meaning of the data and the behaviors of the applications. These changes improve the quality of the design." (Vincenzo Deufemia et al, "Evolutionary Database: State of the Art and Issues", 2009)

"(1) Refactoring (noun): A change made to the internal structure of software to make it easier to understand and cheaper to modify without changing its observable behavior. (2) Refactor (verb): To restructure software by applying a series of refactorings without changing its observable behavior." (Jay Fields et al, Refactoring: Ruby Edition, 2010)

"Refactoring is the process of changing a software system in such a way that it does not alter the external behavior of the code yet improves its internal structure." (Jay Fields et al, Refactoring: Ruby Edition, 2010)

"A (software) refactoring is an explicit, replicable, and beneficial activity that transforms the structure or representation of a software component without changing its meaning (i.e., behavior). The goal of software refactoring is the improvement of the quality (e.g., maintainability) of the software system." (Jörg Rech et al, "Knowledge Patterns" [in "Encyclopedia of Knowledge Management" 2nd Ed.], 2011)

"Restructuring a program to make it more reliable, easier to read, or easier to maintain without changing its outward behavior." (Rod Stephens, "Stephens' Visual Basic® Programming 24-Hour Trainer", 2011)

"Reorganizing code to make it better suited for some purpose, such as parallelization." (Michael McCool et al, "Structured Parallel Programming", 2012)

"The act of changing code to improve some aspect of it, such as how it does something, without changing what it does." (Mark C Lewis, "Introduction to the Art of Programming Using Scala", 2012)

"A process by which existing code or processes are reengineered or restructured to improve nonfunctional aspects without impacting outcomes. It is usually done to make existing assets more manageable, efficient, and maintainable." (Evan Stubbs, "Delivering Business Analytics: Practical Guidelines for Best Practice", 2013)

"To restructure software code without altering its behavior for the purpose of improving quality attributes, easing future extension or adaptation, or adhering to an architectural style." (Project Management Institute, "Software Extension to the PMBOK Guide" 5th Ed, 2013)

"The process of rearranging and rewriting code to make it easier to understand, debug, and maintain." (Rod Stephens, "Beginning Software Engineering", 2015)

"A product quality technique whereby the design of a product is improved by enhancing its maintainability and other desired attributes without altering its expected behavior." (PMI, "A Guide to the Project Management Body of Knowledge" 6th Ed., 2017)

"Refactoring is the activity of improving the internal structure or operation of a code or component without changing its external behavior." (Dean Leffingwell, "SAFe 4.5 Reference Guide: Scaled Agile Framework for Lean Enterprises" 2nd Ed., 2018)

Martin Fowler - Collected Quotes

"It is commonly said that a pattern, however it is written, has four essential parts: a statement of the context where the pattern is useful, the problem that the pattern addresses, the forces that play in forming a solution, and the solution that resolves those forces. [...] it supports the definition of a pattern as 'a solution to a problem in a context', a definition that [unfortunately] fixes the bounds of the pattern to a single problem-solution pair. (Martin Fowler, "Analysis Patterns: Reusable Object Models", 1997)

"Any fool can write code that a computer can understand. Good programmers write code that humans can understand." (Martin Fowler, "Refactoring: Improving the Design of Existing Code", 1999)

"When you find you have to add a feature to a program, and the program's code is not structured in a convenient way to add the feature, first refactor the program to make it easy to add the feature, then add the feature." (Martin Fowler, "Refactoring: Improving the Design of Existing Code", 1999)

"Before you start refactoring, check that you have a solid suite of tests. These tests must be self-checking." (Martin Fowler et al, "Refactoring: Improving the Design of Existing Code", 2002)

"One problem area for refactoring is databases. Most business applications are tightly coupled to the database schema that supports them. That's one reason that the database is difficult to change. Another reason is data migration. Even if you have carefully layered your system to minimize the dependencies between the database schema and the object model, changing the database schema forces you to migrate the data, which can be a long and fraught task." (Martin Fowler et al, "Refactoring: Improving the Design of Existing Code", 2002)

"People also underestimate the time they spend debugging. They underestimate how much time they can spend chasing a long bug. With testing, I know straight away when I added a bug. That lets me fix the bug immediately, before it can crawl off and hide. There are few things more frustrating or time wasting than debugging. Wouldn't it be a hell of a lot quicker if we just didn't create the bugs in the first place?" (Martin Fowler, 2002)

"Refactoring is risky. It requires changes to working code that can introduce subtle bugs. Refactoring, if not done properly, can set you back days, even weeks. And refactoring becomes riskier when practiced informally or ad hoc." (Erich Gamma, 2002)

"Refactoring is the process of changing a software system in such a way that it does not alter the external behavior of the code yet improves its internal structure. It is a disciplined way to clean up code that minimizes the chances of introducing bugs. In essence when you refactor you are improving the design of the code after it has been written." (Martin Fowler et al, "Refactoring: Improving the Design of Existing Code", 2002)

"Refactoring changes the programs in small steps. If you make a mistake, it is easy to find the bug." (Martin Fowler et al, "Refactoring: Improving the Design of Existing Code", 2002)

"Without refactoring, the design of the program will decay. As people change code - changes to realize short-term goals or changes made without a full comprehension of the design of the code - the code loses its structure. It becomes harder to see the design by reading the code. Refactoring is rather like tidying up the code. Work is done to remove bits that aren't really in the right place. Loss of the structure of code has a cumulative effect. The harder it is to see the design in the code, the harder it is to preserve it, and the more rapidly it decays. Regular refactoring helps code retain its shape." (Martin Fowler et al, "Refactoring: Improving the Design of Existing Code", 2002)

"There are many things that make software development complex. But the heart of this complexity is the essential intricacy of the problem domain itself. If you’re trying to add automation to complicated human enterprise, then your software cannot dodge this complexity - all it can do is control it." (Martin Fowler, [forward] 2003)

"The key to controlling complexity is a good domain model, a model that goes beyond a surface vision of a domain by introducing an underlying structure, which gives the software developers the leverage they need. A good domain model can be incredibly valuable, but it’s not something that’s easy to make. Few people can do it well, and it’s very hard to teach." (Martin Fowler, [forward] 2003)

"Graphical design notations have been with us for a while [...] their primary value is in communication and understanding. A good diagram can often help communicate ideas about a design, particularly when you want to avoid a lot of details. Diagrams can also help you understand either a software system or a business process. As part of a team trying to figure out something, diagrams both help understanding and communicate that understanding throughout a team. Although they aren't, at least yet, a replacement for textual programming languages, they are a helpful assistant." (Martin Fowler, "UML Distilled: A Brief Guide to the Standard Object Modeling", 2004)

"Often designers do complicated things that improve the capacity on a particular hardware platform when it might actually be cheaper to buy more hardware." (Martin Fowler, "Patterns of Enterprise Application Architecture", 2012)

"Refactoring is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior. Its heart is a series of small behavior preserving transformations. Each transformation (called a 'refactoring') does little, but a sequence of transformations can produce a significant restructuring. Since each refactoring is small, it's less likely to go wrong. The system is also kept fully working after each small refactoring, reducing the chances that a system can get seriously broken during the restructuring." (Martin Fowler)

Kent Beck - Collected Quotes

"Any fool can write code that a computer can understand. Good programmers write code that humans can understand." (Kent Beck, "Refactoring: Improving the Design of Existing Code", 1999)

"Change is not necessarily slow. A team eager or desperate for improvement can progress quickly. It doesn't need to wait long to assimilate one change before moving on to the next practice. If you change too fast, though, you risk slipping back into old practices and values. When this happens, take time to regroup. Remind yourself of the values you want to hold. Review your practices and remind yourself why you chose them. New habits take time to solidify." (Kent Beck, "Extreme Programming Explained: Embrace Change", 1999)

"Folk wisdom in software development teaches that interfaces shouldn't be unduly influenced by implementations. Writing a test first is a concrete way to achieve this separation." (Kent Beck, "Extreme Programming Explained: Embrace Change", 1999)

"Given the choice between an extremely skilled loner and a competent-but-social programmer, XP teams consistently choose the more social candidate. The best interviewing technique is to have the candidate work with the team for a day. Pair programming provides an excellent test of technical and social skills." (Kent Beck, "Extreme Programming Explained: Embrace Change", 1999)

"If there are forms of testing, like stress and load testing, that find defects after development is 'complete', bring them into the development cycle. Run load and stress tests continuously and automatically." (Kent Beck, "Extreme Programming Explained: Embrace Change", 1999)

"In software development, 'perfect' is a verb, not an adjective. There is no perfect process. There is no perfect design. There are no perfect stories. You can, however, perfect your process, your design, and your stories." (Kent Beck, "Extreme Programming Explained: Embrace Change", 1999)

"Inaccurate estimates are a failure of information, not of values or principles. If the numbers are wrong, fix the numbers and communicate the consequences." (Kent Beck, "Extreme Programming Explained: Embrace Change", 1999)

"Pair programmers: Keep each other on task. Brainstorm refinements to the system. Clarify ideas. Take initiative when their partner is stuck, thus lowering frustration. Hold each other accountable to the team's practices." (Kent Beck, "Extreme Programming Explained: Embrace Change", 1999)

"Responsibility cannot be assigned; it can only be accepted. If someone tries to give you responsibility, only you can decide if you are responsible or if you aren't." (Kent Beck, "Extreme Programming Explained: Embrace Change", 1999)

"Simplicity only makes sense in context." (Kent Beck, "Extreme Programming Explained: Embrace Change", 1999)

"The difference between what I think is valuable and what is really valuable creates waste." (Kent Beck, "Extreme Programming Explained: Embrace Change", 1999)

"The XP philosophy is to start where you are now and move towards the ideal. From where you are now, could you improve a little bit?" (Kent Beck, "Extreme Programming Explained: Embrace Change", 1999)

"Often you'll see the same three or four data items together in lots of places: fields in a couple of classes, parameters in many method signatures. Bunches of data that hang around together really ought to be made into their own object." (Kent Beck, "Refactoring: Improving the Design of Existing Code", 1999)

"When you feel the need to write a comment, first try to refactor the code so that any comment becomes superfluous." (Kent Beck, "Refactoring: Improving the Design of Existing Code", 1999)

"When you find you have to add a feature to a program, and the program's code is not structured in a convenient way to add the feature, first refactor the program to make it easy to add the feature, then add the feature."  (Kent Beck, "Refactoring: Improving the Design of Existing Code", 1999)

"Write contracts for software development that fix time, costs, and quality but call for an ongoing negotiation of the precise scope of the system. Reduce risk by signing a sequence of short contracts instead of one long one." (Kent Beck, "Extreme Programming Explained: Embrace Change", 1999)

"One of the purposes of planning is we always want to work on the most valuable thing possible at any given time. We can’t pick features at random and expect them to be most valuable. We have to begin development by taking a quick look at everything that might be valuable, putting all our cards on the table. At the beginning of each iteration the business (remember the balance of power) will pick the most valuable features for the next iteration." (Kent Beck & Martin Fowler, "Planning Extreme Programming", 2000)

"Optimism is an occupational hazard of programming: feedback is the treatment." (Kent Beck, "Extreme Programming Explained", 2000)

"Planning is not about predicting the future. When you make a plan for developing a piece of software, development is not going to go like that. Not ever. Your customers wouldn’t even be happy if it did, because by the time software gets there, the customers don’t want what was planned, they want something different." (Kent Beck & Martin Fowler, "Planning Extreme Programming", 2000)

"Projects sometimes fail long before they deliver anything. At some point they may be determined to be too expensive to continue. Or perhaps they took too long to develop and the business need evaporated. Or perhaps the requirements change so often that the developers can never finish one thing without having to stop and start all over on something new. Certainly these are planning failures." (Kent Beck & Martin Fowler, "Planning Extreme Programming", 2000)

"The business changes. The technology changes. The team changes. The team members change. The problem isn't change, per se, because change is going to happen; the problem, rather, is the inability to cope with change when it comes." (Kent Beck, Extreme Programming Explained, 2000)

"The new concept of Extreme Programming (XP) is gaining more and more acceptance, partially because it is controversial, but primarily because it is particularly well-suited to help the small software development team succeed. [...] XP is controversial, many software development sacred cows don't make the cut in XP; it forces practitioners to take a fresh look at how software is developed." (Kent Beck, "Abstract Extreme Programming Explained", 2000)

"There are two ways to approach prevention of these planning failures. We can plan not to lose, or we can plan to win. The two are not identical. Planning not to lose is defensive; while planning to win is aggressive.[...] the problem that planning is supposed to solve is simply, to build the right system at the right cost. If we take a defensive posture by planning not to lose, we will be able to hold people accountable for any failures; but at an enormous cost. If we take an aggressive posture and plan to win, we will be unafraid to make errors, and will continuously correct them to meet our goals." (Kent Beck & Martin Fowler, "Planning Extreme Programming", 2000)

"We plan because: We need to ensure that we are always working on the most important thing we need to do. We need to coordinate with other people. When unexpected events occur we need to understand the consequences for the first two." (Kent Beck & Martin Fowler, "Planning Extreme Programming", 2000)

"When we plan to win we take direct steps to ensure that we are building the right system at the best possible cost. Every action we take goes towards that end. Instead of trying to plan everything up front, we plan just the next few steps; and then allow customer feedback to correct our trajectory. In this way, we get off the mark quickly, and then continuously correct our direction. Errors are unimportant because they will be corrected quickly." (Kent Beck & Martin Fowler, "Planning Extreme Programming", 2000)