28 September 2008

W3: Semantic Web (Definitions)

"The Web of data with meaning in the sense that a computer program can learn enough about what the data  means to process it." (Tim Berners-Lee, "Weaving the Web", 1999)

"An evolving, collaborative effort led by the W3C whose goal is to provide a common framework that will allow data to be shared and re-used across various applications as well as across enterprise and community boundaries." (J P Getty Trust, "Introduction to Metadata" 2nd Ed, 2008)

"Communication protocols and standards that would include descriptions of the item on the Web such as people, documents, events, products, and organizations, as well as, relationship between documents and relationships between people." (Craig F Smith & H Peter Alesso, "Thinking on the Web: Berners-Lee, Gödel and Turing", 2008)

"The Web of data with meaning in the sense that a computer program can learn enough about what the data means to process it. The principle that one should represent separately the essence of a document and the style is presented." (Craig F Smith & H Peter Alesso, "Thinking on the Web: Berners-Lee, Gödel and Turing", 2008)

"A machine-processable web of smart data, [where] smart data is data that is application-independent, composeable, classified, and part of a larger information ecosystem (ontology)." (David C Hay, "Data Model Patterns: A Metadata Map", 2010)

"An evolving extension of the Web in which Web content can be expressed not only in natural language but also in a form that can be understood, interpreted, and used by intelligent computer software agents, permitting them to find, share, and integrate information more easily." (Linda Volonino & Efraim Turban, "Information Technology for Management" 8th Ed., 2011)

"The next-generation Internet in which all content is tagged with semantic tags defined in published ontologies. Interlinking these ontologies will allow software agents to reason about information not directly connected by document creators." (DAMA International, "The DAMA Dictionary of Data Management", 2011)

"is a term coined by World Wide Web Consortium (W3C) director Sir Tim Berners-Lee. It describes methods and technologies to allow machines to understand the meaning - or 'semantics'- of information on the World Wide Web." (Jingwei Cheng et al, "RDF Storage and Querying: A Literature Review", 2016)

"The vision of a Semantic Web world builds upon the web world, but adds some further prescriptions and constraints for how to structure descriptions. The Semantic Web world unifies the concept of a resource as it has been developed in this book, with the web notion of a resource as anything with a URI. On the Semantic Web, anything being described must have a URI. Furthermore, the descriptions must be structured as graphs, adhering to the RDF metamodel and relating resources to one another via their URIs. Advocates of Linked Data further prescribe that those descriptions must be made available as representations transferred over HTTP." (Robert J Glushko, "The Discipline of Organizing: Professional Edition" 4th Ed., 2016)

"A collaborative effort to enable the publishing of semantic machine-readable and shareable data on the Web." (Panos Alexopoulos, "Semantic Modeling for Data", 2020)

16 September 2008

W3: Cyberspace (Definitions)

"A term used to describe the nonphysical, virtual world of computers." (Andy Walker, "Absolute Beginner’s Guide To: Security, Spam, Spyware & Viruses", 2005)

"A metaphoric abstraction for a virtual reality existing inside computers and on computer networks." (DAMA International, "The DAMA Dictionary of Data Management", 2011)

"The online world of computer networks where people can interact with others without physically being with them. People commonly interact with cyberspace via the Internet." (Darril Gibson, "Effective Help Desk Specialist Skills", 2014)

"The interdependent network of information technology infrastructures, which includes the Internet, telecommunications networks, computer systems, and embedded processors and controllers." (Olivera Injac & Ramo Šendelj, "National Security Policy and Strategy and Cyber Security Risks", 2016)

"A complex hyper-dimensional space involving the state of many mutually dependent computer and network systems with complex and often surprising properties as compared to physical space." (O Sami Saydjari, "Engineering Trustworthy Systems: Get Cybersecurity Design Right the First Time", 2018)

"Artifacts based on or dependent on computer and communications technology; the information that these artifacts use, store, handle, or process; and the interconnections among these various elements." (William Stallings, "Effective Cybersecurity: A Guide to Using Best Practices and Standards", 2018)

"Refers to a physical and non-physical terrain created by and/or composed of some or all of the following: computers, computer systems, networks, and their computer programs, computer data, content data, traffic data, and users." (Thokozani I Nzimakwe, "Government's Dynamic Approach to Addressing Challenges of Cybersecurity in South Africa", 2018)

"Cyberspace, is supposedly 'virtual' world/network created by links between computers, Internet-enabled devices, servers, routers, and other components of the Internet’s infrastructure." (Sanjeev Rao et al, "Online Social Networks Misuse, Cyber Crimes, and Counter Mechanisms", 2021)

31 August 2008

SQL Server New Features: ROWCOUNT in action

 Especially when working with big tables, the default behaviour of Query Analyzer is to not show the output until the last record has been fetched. This can be time and resource consuming and therefore I’ve appreciated the fact that TOAD and SQL Developer are fetching only a certain number of records. Now I can see that same can be done starting with SQL Server 2005 onward by modifying ROWCOUNT server property using Query/Query Options menu functionality.

  Query Options under SQL Server 2008 Query Options under SQL Server 2008 or by running the command: SET ROWCOUNT <number of records>; Of course somebody may limit the number of records returned by a query using TOP function when working with SQL Server and ROWNUM in Oracle, though I find it not always handy – it depends from case to case. There are also technical implications between the two types of usage, according SQL Server Books online it is recommended to TOP with SELECT over ROWCOUNT with regard to scope and query optimization, however in this context only the later makes sense:
"As a part a SELECT statement, the query optimizer can use the value of expression in the TOP clause as part of generating an execution plan for a query. Because SET ROWCOUNT is used outside a statement that executes a query, its value cannot be used to generate a query plan for a query."
Notes: 1. Do not mix the ROWNUM with @@ROWNUM function which returns the number of rows affected by the last statement. 2. Some of us list all the records in order to see the number of records returned by a query, though that’s totally not advisable!

AdventureWorks requires FILESTREAM enabled

 Surprises, surprises, surprises, programmers’ world is full of them! When you say that everything is ok, you just discover that something went wrong. I was expecting to have Adventure Works database installed though I haven’t checked that. I realized today that it’s missing, so I tried to reinstall it enabling this time the “Restore AdventureWorks DBs” feature, though I got another nice error:

  Setup failed for MSSQLSERVER. The following features are missing: FILESTREAM Fix the problems and re-run setup.

 Guy Burstein, in his blog, wrote that the STREAM support can be enabled using the following SQL command: exec [dbo.sp_filestream_configure] @enable_level = 3; I tried that and another error came in:

  Msg 2812, Level 16, State 62, Line 1 Could not find stored procedure 'sp_filestream_configure'

 Checking my local installation of SQL Server Books Online, I found no track of sp_filestream_configure stored procedure, but I found that I can enable the STREAM support using sp_configure stored procedure as below:

EXEC sp_configure filestream_access_level, 2
RECONFIGURE
GO

 Once I executed the 3 lines together, I got the following confirmation message which, amusingly, still recommands me to run the RECONFIGURE statement even if I did that. Anyway better more redundant information than nothing…

  Configuration option 'filestream access level' changed from 2 to 2. Run the RECONFIGURE statement to install.

30 August 2008

Oracle Troubleshooting: No records returned by queries (Checklist)

No records returned by a query even if there should be results? Usually I’m using the following checklist: 1. check if the tables contain data. Silly but effective, especially in Oracle APPS in which some tables got deprecated and were replaced by tables with similar names (PA_PROJECTS_ALL vs. PA_PROJECTS), though that could happen in other environments too; 
2. check if the JOIN syntax is correct; 
3. check if one of the columns use in JOIN has only NULL values; 
4. check if the constraints used in WHERE clause causes makes sense (e.g. wrong values or syntax); 
5. for Oracle flavored queries, check if in WHERE clause there is a column not referenced with the table name or alias, and the column is available in more than one table used in the query. This Oracle bug is really dangerous when doing fast query checks! 
6. for Oracle (APPS), check whether the query or view uses USERENV function with LANG or LANGUAGE text parameter, normally a constraint like: TABLE1.LANGUAGE = USERENV(‘LANG’).
The problem with such queries comes when user’s system language is other than the one expected, and thus query’s output might not be as expected. Usually it is preferable to hardcode the value, when possible: TABLE1.LANGUAGE = ‘US’ Note: Actually, also the tools you are using to run a query could create issues, for example a query run under Oracle’s SQL Developer was not returning records even if in TOAD did that. The problem was solved with the installation of a newer SQL Developer version.

Oracle Troubleshooting: ANSI 92 JOIN syntax error

 Lately I’ve been working a lot with Oracle APPS, doing mainly ad-hoc reporting. One of my nightmares is an Oracle bug related to ANSI 92 syntax:

  “ORA-01445: cannot select ROWID from, or sample, a join without a key-preserved table”

 Unfortunately, even if the bug was solved by Oracle, it seems the update was missed on some servers and the bug haunts my queries almost on a daily basis. 

 Having an SQL Server background and, for code clearness, I prefer ANSI 92 JOIN syntax:

SELECT A.column1, B.column2
FROM table1 A JOIN table2 B
 ON A.column1 = B.column2 

instead of using the old fashioned writing:

SELECT A.column1, B.column2
FROM table1 A , table2 B
WHERE A.column1 = B.column2

 In theory the two queries should provide the same output and have, hopefully, similar performance. The problem with ANSI 92 syntax is that, on some Oracle installations, when the number of joins exceeds a certain limit, usually greater than 7, the above error is thrown.

What one can do is to reduce the number of joins to the main table by restructuring the query and grouping multiple tables into subqueries, which are then joined to the main table. For the tables from which is returned only one column, one can move the table into the SELECT statement.

Happy coding! 

💠🛠️SQL Server: Administration (Part III: Troubleshooting Adventure Works installation error on Vista)

I tried to install the Adventure Works OLTP & DW on SQL Server 2008 RTM from CodePlex though I got an error:

“The installer has encountered an unexpected error installing this package. This may indicate a problem with this package. The error code is 2738.”

Initially I thought that the problem was caused by existing Adventure Works installations made on a previous CTP version of SQL Server 2008, forgetting to uninstall them when I uninstalled the CTP version. Totally wrong! Doing a little research, I found first a post on CodePlex Discussion forum mentioning that the problem could be caused by VBScript runtime because, as Toms’Tricks blog highlights, VBScript and Jscript are not registered on Windows Vista. Wonderful! I just run regsvr32 vbscript.dll command and it worked! Another situation in which regsvr32 saved the day!

I wonder why I haven’t got the same problem when I previously installed Adventure Works database on CTP version! Could it be because of Windows Vista SP1 changes (I installed Windows Vista SP1 after SQL Server CTP)?

💠🛠️SQL Server: Administration (Part II: Troubleshooting Microsoft SQL Server 2008 installation error)

 This week I tried to install SQL Server 2008 however I got the following error: 
  “A previous release of Microsoft Visual Studio 2008 is installed on this computer. Upgrade Microsoft Visual Studio 2008 to the SP1 before installing SQL Server 2008.” 

 It’s true that I have previously installed Visual Studio 2008, though once I did that I checked if there are any updates and thus I installed SP1 too. I did a quick search on Google and the first results pointed me to an article o Microsoft Help and Support website: Visual Studio 2008 SP1 may be required for SQL Server 2008 installations. It didn’t make sense; in the end I’ve installed the server but enabled the installation of the following components: 
 • Management Tools (Basic or Complete) 
 • Integration Services 
 • Business Intelligence Development Studio 

 The server was installed without problems, so I tried to install the remaining components getting the same error as above. I had to stop at that point and today, giving more thought to the problem, I realized that the error could be caused by Microsoft Visual Studio 2008 Express edition, which I managed to install a few months back. Instead of uninstalling Microsoft Visual Studio 2008 it looked easier to uninstall the Express version, and once I did that, I managed to install the remaining components. Actually I checked before if there is a SP1 for Microsoft Visual Studio 2008 Express, I arrived at Microsoft Visual Studio 2008 Express Editions with SP1 page, though I remembered that I have to install the Web Developer, Visual Basic and C# 2008 separately and in the end I presumed that maybe it would be easier to uninstall the existing versions and try then to install SQL Server remaining components. I haven’t tried to install the Express editions with SP1 as now I have the Professional edition.

04 August 2008

Application Architecture: Enterprise Service Bus (Definitions)

"A layer of middleware that enables the delivery and sharing of services across and between business applications. ESBs are typically used to support communication, connections, and mediation in a service-oriented architecture." (Evan Levy & Jill Dyché, "Customer Data Integration", 2006)

"The infrastructure of a SOA landscape that enables the interoperability of services. Its core task is to provide connectivity, data transformations, and (intelligent) routing so that systems can communicate via services. The ESB might provide additional abilities that deal with security, reliability, service management, and even process composition. However, there are different opinions as to whether a tool to compose services is a part of an ESB or just an additional platform to implement composed and process services outside the ESB." (Nicolai M Josuttis, "SOA in Practice", 2007)

"A middleware software architecture construct that provides foundational services for more complex architectures via an event-driven and standards-based messaging engine (the bus). An ESB generally provides an abstraction layer on top of an implementation of an enterprise messaging system. |" (Alex Berson & Lawrence Dubov, "Master Data Management and Data Governance", 2010)

"The infrastructure of an SOA landscape that enables the interoperability of services. Its core task is to provide connectivity, data transformations, and routing so that systems can communicate via services." (David Lyle & John G Schmidt, "Lean Integration", 2010)

"A software layer that provides data between services on an event-driven basis, using standards for data transmission between the services." (DAMA International, "The DAMA Dictionary of Data Management", 2011)

"A packaged set of middleware services that are used to communicate between business services in a secure and predictable manner." (Marcia Kaufman et al, "Big Data For Dummies", 2013)

22 February 2008

Business Applications: Customer Relationship Management (Definitions)

"Operational and analytic processes that focus on better understanding and servicing customers in order to maximize mutually beneficial relationships with each customer." (Ralph Kimball & Margy Ross, "The Data Warehouse Toolkit" 2nd Ed., 2002)

"a popular DSS application designed to streamline customer and/or corporate relationships." (William H Inmon, "Building the Data Warehouse", 2005)

"A database system containing information on interactions with customers." (Glenn J Myatt, "Making Sense of Data", 2006)

"The infrastructure that enables the delineation of and increase in customer value, and the correct means by which to increase customer value and motivate valuable customers to remain loyal, indeed, to buy again." (Jill Dyché & Evan Levy, Customer Data Integration, 2006)

"The tracking and management of all the organization’s interactions with its customers in order to provide better service, encourage customer loyalty, and increase the organization’s long-term profit per customer." (Steve Williams & Nancy Williams, "The Profit Impact of Business Intelligence", 2007)

"A strategy devoted to the development and management of close relationships between customers and the company. In many cases, CRM is referred to as the automation tool that helps bring about this strategy." (Steven Haines, "The Product Manager's Desk Reference", 2008)

"Software intended to help you run your sales force and customer support." (Judith Hurwitz et al, "Service Oriented Architecture For Dummies" 2nd Ed., 2009)

"The technology and processes used to capture the details of interactions with customers and analyze that data to improve customer interaction, assess customer value, and build value and further loyalty." (Tony Fisher, "The Data Asset", 2009)

"A set of technologies and business processes designed to understand a customer, improve customer experience, and optimize customer-facing business processes across marketing, sales, and servicing channels." (Alex Berson & Lawrence Dubov, "Master Data Management and Data Governance", 2010)

"Refers to the set of procedures and computer applications designed to manage and improve customer service in an enterprise. Data warehousing, with integrated data about each customer, is eminently suitable for CRM." (Paulraj Ponniah, "Data Warehousing Fundamentals for IT Professionals", 2010)

"This is a packaged solution that delivers an end-to-end solution around contacting, understanding, and serving particular customer needs." (Martin Oberhofer et al, "The Art of Enterprise Information Architecture", 2010)

"Establishing relationships with individual customers and then using that information to treat different customers differently. Customer buying profiles and churn analysis are examples of decision support activities that can affect the success of customer relationships. Effective CRM is dependent on high quality master data about individuals and organizations (customer data integration)." (DAMA International, "The DAMA Dictionary of Data Management", 2011)

"The entire process of maximizing the value proposition to the customer through all interactions, both online and traditional. Effective CRM advocates one-to-one relationships and participation of customers in related business decisions." (Linda Volonino & Efraim Turban, "Information Technology for Management" 8th Ed., 2011)

"Software designed to help you run your sales force and customer support operations." (Marcia Kaufman et al, "Big Data For Dummies", 2013)

"The management of current and future customer interactions with a business. This can include sales support, warranty and technical support activity, Internet website, marketing, and product advertising." (Kenneth A Shaw, "Integrated Management of Processes and Information", 2013)

"Customer relationship management is a model for managing a company’s interactions with current and future customers. It involves using technology to organize, automate, and synchronize sales, marketing, customer service, and technical support." (Keith Holdaway, "Harness Oil and Gas Big Data with Analytics", 2014)

"System that manages the customer-related data along with their past, present, and future interactions with the organization." (Hamid R Arabnia et al, "Application of Big Data for National Security", 2015)

"A set of tools, techniques, and methodologies for understanding the needs and characteristics of customers in order to better serve them." (Robert M Grant, "Contemporary Strategy Analysis" 10th Ed., 2018)

"CRM is the practice of helping customers manage sales, service, and marketing processes effectively and efficiently so that they can grow their business and provide excellent customer service." (Srini Munagavalasa, "The Salesforce Business Analyst Handbook", 2022)

13 January 2008

💎SQL Reloaded: Advices on SQL logic split in Web Applications

Yesterday I remembered about my first two small fights on SQL related topics, though in both situations I had to renounce temporarily to my opinions for the sake of armistice. It’s understandable, we can all make mistakes, unfortunately what we know hurts us more then what we don’t know. It’s amusing when I’m thinking about the two issues in discussion at those times, though then it was a little disturbing. What was about?! Actually both issues are related to web applications, my first “professional encounter” with programming.

Issue 1 – using JOINs or backend vs middle tier processing

JOINs are a powerful feature of SQL in combing related data from multiple tables in only one query, this coming with a little (or more) overhead from the database server side.
Web applications make use of lot of data access operations, data being pulled from a database each time a user requests a page, of course that happening when the page needs data from database(s) or execute commands on it, the CRUD (Create/Read/Update/Delete) gamma. That can become costly in time, depending on how data access was architected and requirements. The target is to pull smallest chunk of data possible (rule 1), with a minimum of trips to the database (rule 2).

Supposing that we need Employees data from a database for a summary screen with all employees, it could contain First Name, Last Name, Department and Contact information – City, Country, Email Address and Phone Number. Normally the information could be stored in 4 tables – Employees, Departments, Address and Countries, like in the below diagram.


The easiest and best way to pull the Employee needed data is to do a JOIN between tables:

-- Employee details
SELECT E.EmployeeId
, E.FirstName 
, E.LastName 
, D.Department 
, A.City 
, C.Country 
, A.Phone 
, E.EmailAddress
FROM dbo.Employees E
     JOIN dbo.Departments D
	   ON E.DepartmentId = D.DepartmentId 
	 JOIN dbo.Addresses A
	   ON A.EmployeeId = A.EmployeeId 
	      JOIN dbo.Countries C
		    ON A.CountryId = C.CountryId 

More likely that two or more employees will have the same country or department, resulting in “duplication” of small pieces of information within the whole data set, contradicting rule 1. Can be pulled smaller chunks of data targeting only the content of a table, that meaning that we have to pull first all Employees or the ones matching a set of constraints, then all the departments or the only the ones for which an Employee was returned, and same with Addresses and Countries. In the end will have 4 queries and same number of roundtrips (or more). In the web page the code will have to follow the below steps:

Step 1: Pull the Employee data matching the query:
SELECT E.EmployeeID
, E.DepartmentID
, E.FirstName
, E.LastName
, E.EmailAddress
FROM Employees E
WHERE 

Step 2: Build the (distinct) list of Department IDs and a (distrinct) list of Employee IDs.

Step 3: Pull the Department data matching the query:
SELECT D.DepartmentID
, D.Department
FROM Departments D
WHERE DepartmentID IN (<list of Department IDs>)


Step 4: Pull the Address data matching the query:
SELECT A.EmployeeID
, A.CountryID
, A.City
, A.Phone
FROM Addresses A
WHERE EmployeeID IN (<list of Employee IDs>)


Step 5: Build the (distinct) list of Country IDs.

Step 6: Pull the Country data matching the query:
SELECT C.CountryID
, C.Country
FROM Countries C
WHERE CountryID IN (<list of Country IDs>)


And if this doesn’t look like an overhead for you, you have to take into account that for each Employee is needed to search the right Department from the set of data returned in Step 3, and same thing for Addresses and Countries. It’s exactly what the database server does but done on the web server, with no built in capabilities for data matching.
In order to overcome the problems raised by matching, somebody could go and execute for each employee returned in Step 1 a query like the one defined in Step 4, but limited only to the respective Employee, thus resulting an additional number of new roundtrips matching the number of Employees. Quite a monster, isn’t it? Please don’t do something like this!

It’s true that we always need to mitigate between minimum of data and minimum of roundtrips to a web server, though we have to take into account also the overhead created by achieving extremities and balance them in an optimum manner, implementing the logic on the right tier. So, do data matching as much as possible on the database server because it was designed for that, and do, when possible, data enrichment (e.g. formatting) only on the web server.

In theory the easiest way of achieving something it’s the best as long the quality remains the same, so try to avoid writing expensive code that’s hard to write, maintain and debug!

Issue 2: - LEFT vs FULL JOINs

Normally each employee should be linked to a Department, have at least one Address, and the Address should be linked to a Country. That can be enforced at database and application level, though it’s not always the case. There could be Employees that are not assigned to a Department, or without an Address; in such cases then instead of a FULL JOIN you have to consider a LEFT or after case a RIGHT (OUTER) JOIN. So, I’ve rewritten the first query, this time using LEFT JOINs.

-- Employee details (with LEFT JOINs)
SELECT E.EmployeeId
, E.FirstName 
, E.LastName 
, D.Department 
, A.City 
, C.Country 
, A.Phone 
, E.EmailAddress
FROM dbo.Employees E
     LEFT JOIN dbo.Departments D
	   ON E.DepartmentId = D.DepartmentId 
	 LEFT JOIN dbo.Addresses A
	   ON A.EmployeeId = A.EmployeeId 
	      LEFT JOIN dbo.Countries C
		    ON A.CountryId = C.CountryId 

Important:
Don't use LEFT JOINs unless the business case requires it, and don’t abuse of them as they can come with performance penalties!7

12 January 2008

💎SQL Reloaded: SQL Server and Excel Data

Looking after information about SQL Server 2008, I stumble over Bob Beauchemin’s blog, the first posting I read Another use for SQL Server 2008 row constructors demonstrating a new use of VALUES clause that allows to insert multiple lines in a table by using only one query, or to group a set of values in a table and use them as source for a JOIN. I was waiting for this feature to be available under SQL Server 2005, though better later than never!

The feature is useful when you need to limit the output of a query based on matrix (tabular) values coming from an Excel or text file. For exemplification I’ll use HumanResources.vEmployee view from AdventureWorks database that co/mes with SQL Server 2005, you can download it from Code Plex in case you don’t have it for SQL Server 2008.
Let’s suppose that you have an Excel file with Employees for which you need contact information from a table available on SQL Server. You have the FirstName, MiddeName and LastName, and you need the EmailAddress and Phone. In SQL Server 2008 you can do that by creating a temporary table-like structure on the fly using VALUES clause, and use it then in JOIN or INSERT statements.

Query 1

The heart of the query is the below structure, where B(FirstName, MiddleName, LastName) is the new table, each row in its definition being specified by comma delimited triples of form ('FirstName ', 'MiddleName ', ' LastName'):



The construct it’s time consuming to build manually, especially when the number of lines is considerable big, though you can get the construct in Excel with the help of an easy formula.



The formula from column D is = ", ('" & A2 & "','" & B2 & "','" & C2 & "')" and it can be applied to the other lines too. You just need to copy now the data from Column D to SQL Server and use them in Query with a few small changes. Of course, you can create also a custom function (macro) in Excel to obtain the whole structure is a singe cell.

You can do something alike under older versions of SQL Server (or other databases) using a simple trick – concatenating the values from each column by row by using a delimiter like “/”, “~”, “|” or any other delimiter, though you have to be sure that the delimiter isn’t found in your data sources (Excel and table). Using “/” the formula is = ", '" & A2 & "/" & B2 & "/" & C2 & "'".



Then you have to use the same trick and concatenate the columns from the table, the query becoming:

Query 2

This technique involves small difficulties when:
• The data used for searching have other type than string derived data types, however that can be overcome by casting the values to string before concatenation.
• The string values contain spaces at extremities, so it’s better to trim the values using LTrim and RTrim functions.
• The values from the two sources are slightly different, for example diacritics vs. Latin standard characters equivalents, for this being necessary a transformation of the values to the same format.

31 December 2007

🏗️Software Engineering: Problem Solving (Just the Quotes)

"We can scarcely imagine a problem absolutely new, unlike and unrelated to any formerly solved problem; but if such a problem could exist, it would be insoluble. In fact, when solving a problem, we should always profit from previously solved problems, using their result or their method, or the experience acquired in solving them." (George Polya, 1945)

"The problems are solved, not by giving new information, but by arranging what we have known since long." (Ludwig Wittgenstein, "Philosophical Investigations", 1953)

"A great many problems are easier to solve rigorously if you know in advance what the answer is." (Ian Stewart, "From Here to Infinity", 1987)

"An important symptom of an emerging understanding is the capacity to represent a problem in a number of different ways and to approach its solution from varied vantage points; a single, rigid representation is unlikely to suffice." (Howard Gardner, "The Unschooled Mind", 1991)

"[By understanding] I mean simply a sufficient grasp of concepts, principles, or skills so that one can bring them to bear on new problems and situations, deciding in which ways one’s present competencies can suffice and in which ways one may require new skills or knowledge." (Howard Gardner, "The Unschooled Mind", 1991) 

"Solving a problem for which you know there’s an answer is like climbing a mountain with a guide, along a trail someone else has laid. In mathematics, the truth is somewhere out there in a place no one knows, beyond all the beaten paths. And it’s not always at the top of the mountain. It might be in a crack on the smoothest cliff or somewhere deep in the valley." (Yōko Ogawa, "The Housekeeper and the Professor", 2003)

"Framing the right problem is equally or even more important than solving it." (Pearl Zhu, "Change, Creativity and Problem-Solving", 2017)

"A great discovery solves a great problem but there is a grain of discovery in the solution of any problem. Your problem may be modest; but if it challenges your curiosity and brings into play your inventive faculties, and if you solve it by your own means, you may experience the tension and enjoy the triumph of discovery." (George Polya)

"A great many problems are easier to solve rigorously if you know in advance what the answer is." (Ian Stewart, "From Here to Infinity", 1987)"Every problem has a solution; it may sometimes just need another perspective.” (Rebecca Mallery et al, “NLP for Rookies”, 2009)

"Mostly we rely on stories to put our ideas into context and give them meaning. It should be no surprise, then, that the human capacity for storytelling plays an important role in the intrinsically human-centered approach to problem solving, design thinking." (Tim Brown, "Change by Design: How Design Thinking Transforms Organizations and Inspires Innovation", 2009)

"One of the broad truths we’ve seen to be true is the idea that finding problems earlier in the developer workflow usually reduces costs." (Titus Winters, "Software Engineering at Google: Lessons Learned from Programming Over Time", 2020)

"A problem thoroughly understood is always fairly simple." (Charles Kettering)

"A problem well-defined is a problem half solved." (John Dewey)

"An expert problem solver must be endowed with two incompatible qualities, a restless imagination and a patient pertinacity." (Howard W Eves)

"Finding the right answer is important, of course. But more important is developing the ability to see that problems have multiple solutions, that getting from X to Y demands basic skills and mental agility, imagination, persistence, patience." (Mary H Futrell)

"I have not seen any problem, however complicated, which, when you looked at it in the right way, did not become still more complicated." (Paul Anderson)

"I knew nothing, except how to think, how to grapple with a problem and then go on grappling with it until you had solved it." (Sir Barnes Wallis)

"Man is not born to solve the problems of the universe, but to find out where the problems begin, and then to take his stand within the limits of the intelligible." (Johann Wolfgang von Goethe)

"One is always a long way from solving a problem until one actually has the answer." (Stephen Hawking) 

"One measure of our understanding is the number of independent ways we are able to get to the same result." (Richard P Feynman) 

“Solving problems is a practical skill like, let us say, swimming. We acquire any practical skill by imitation and practice.” (George Polya)

"Some problems are just too complicated for rational logical solutions. They admit of insights, not answers." (Jerome B Wiesner)

"The best way to escape from a problem is to solve it." (Brendan Francis)

"The greatest challenge to any thinker is stating the problem in a way that will allow a solution." (Bertrand Russell)

"The measure of our intellectual capacity is the capacity to feel less and less satisfied with our answers to better and better problems." (Charles W Churchman)

"The mere formulation of a problem is often far more essential than its solution. To raise new questions, new possibilities, to regard old problems from a new angle, requires creative imagination and marks real advances in science." (Albert Einstein)

"The worst thing you can do to a problem is solve it completely." (Daniel Kleitman)

"There is no such thing as a problem without a gift. We seek problems because we need their gifts." (Richard Bach)

"To ask the right question is harder than to answer it." (Georg Cantor)

"When the answer to a mathematical problem cannot be found, then the reason is frequently that we have not recognized the general idea from which the given problem only appears as a link in a chain of related problems." (David Hilbert) 

"You are never sure whether or not a problem is good unless you actually solve it." (Mikhail Gromov)

🏗️Software Engineering: Architecture (Just the Quotes)

"The term architecture is used here to describe the attributes of a system as seen by the programmer, i.e., the conceptual structure and functional behavior, as distinct from the organization of the data flow and controls, the logical design, and the physical implementation." (Gene Amdahl et al, "Architecture of the IBM System", IBM Journal of Research and Development. Vol 8 (2), 1964)

"In computer design three levels can be distinguished: architecture, implementation and realisation; for the first of them, the following working definition is given: The architecture of a system can be defined as the functional appearance of the system to the user, its phenomenology. […] The inner structure of a system is not considered by the architecture: we do not need to know what makes the clock tick, to know what time it is. This inner structure, considered from a logical point of view, will be called the implementation, and its physical embodiment the realisation." (Gerrit A Blaauw, "Computer Architecture", 1972)

"There always is an architecture, whether it is defined in advance - as with modern computers - or found out after the fact - as with many older computers. For architecture is determined by behavior, not by words. Therefore, the term architecture, which rightly implies the notion of the arch, or prime structure, should not be understood as the vague overall idea. Rather, the product of the computer architecture, the principle of operations manual, should contain all detail which the user can know, and sooner or later is bound to know." (Gerrit A Blaauw, "Computer Architecture", 1972)

"The design of a digital system starts with the specification of the architecture of the system and continues with its implementation and its subsequent realisation... the purpose of architecture is to provide a function. Once that function is established, the purpose of implementation is to give a proper cost-performance and the purpose of realisation is to build and maintain the appropriate logical organisation." (Gerrit A Blaauw, "Specification of Digital Systems", Proc. Seminar in Digital Systems Design, 1978)

"With increasing size and complexity of the implementations of information systems, it is necessary to use some logical construct (or architecture) for defining and controlling the interfaces and the integration of all of the components of the system." (John Zachman, "A Framework for Information Systems Architecture", 1987)

"Every software system needs to have a simple yet powerful organizational philosophy (think of it as the software equivalent of a sound bite that describes the system's architecture). [A] step in [the] development process is to articulate this architectural framework, so that we might have a stable foundation upon which to evolve the system's function points." (Grady Booch, "Object-Oriented Design: with Applications", 1991)

"As the size of software systems increases, the algorithms and data structures of the computation no longer constitute the major design problems. When systems are constructed from many components, the organization of the overall system - the software architecture - presents a new set of design problems. This level of design has been addressed in a number of ways including informal diagrams and descriptive terms, module interconnection languages, templates and frameworks for systems that serve the needs of specific domains, and formal models of component integration mechanisms." (David Garlan & Mary Shaw, "An introduction to software architecture", Advances in software engineering and knowledge engineering Vol 1, 1993)

"Software architecture involves the description of elements from which systems are built, interactions among those elements, patterns that guide their composition, and constraints on these patterns. In general, a particular system is defined in terms of a collection of components and interactions among those components. Such a system may in turn be used as a (composite) element in a larger system design." (Mary Shaw & David Garlan,"Characteristics of Higher-Level Languages for Software Architecture", 1994)

"If a project has not achieved a system architecture, including its rationale, the project should not proceed to full-scale system development. Specifying the architecture as a deliverable enables its use throughout the development and maintenance process." (Barry Boehm, 1995)

"Our experience with designing and analyzing large and complex software-intensive systems has led us to recognize the role of business and organization in the design of the system and in its ultimate success or failure. Systems are built to satisfy an organization's requirements (or assumed requirements in the case of shrink-wrapped products). These requirements dictate the system's performance, availability, security, compatibility with other systems, and the ability to accommodate change over its lifetime. The desire to satisfy these goals with software that has the requisite properties influences the design choices made by a software architect." (Len Bass et al, "Software Architecture in Practice", 1998)

"Generically, an architecture is the description of the set of components and the relationships between them. […] A software architecture describes the layout of the software modules and the connections and relationships among them. A hardware architecture can describe how the hardware components are organized. However, both these definitions can apply to a single computer, a single information system, or a family of information systems. Thus 'architecture' can have a range of meanings, goals, and abstraction levels, depending on who’s speaking." (Frank J Armour et al, "A big-picture look at enterprise architectures", IT professional Vol 1 (1), 1999)

"An architecture framework is a tool which can be used for developing a broad range of different architectures [architecture descriptions]. It should describe a method for designing an information system in terms of a set of building blocks, and for showing how the building blocks fit together. It should contain a set of tools and provide a common vocabulary. It should also include a list of recommended standards and compliant products that can be used to implement the building blocks." (TOGAF, 2002)

"The aim of architectural design is to prepare overall specifications, derived from the needs and desires of the user, for subsequent design and construction stages. The first task for the architect in each design project is thus to determine what the real needs and desires of the user are […]" (George J Klir & Doug Elias, "Architecture of Systems Problem Solving" 2nd Ed, 2003)

"The software architecture of a system or a family of systems has one of the most significant impacts on the quality of an organization's enterprise architecture. While the design of software systems concentrates on satisfying the functional requirements for a system, the design of the software architecture for systems concentrates on the nonfunctional or quality requirements for systems. These quality requirements are concerns at the enterprise level. The better an organization specifies and characterizes the software architecture for its systems, the better it can characterize and manage its enterprise architecture. By explicitly defining the systems software architectures, an organization will be better able to reflect the priorities and trade-offs that are important to the organization in the software that it builds." (James McGovern et al, "A Practical Guide to Enterprise Architecture", 2004)

"The traditional view on software architecture suffers from a number of key problems that cannot be solved without changing our perspective on the notion of software architecture. These problems include the lack of first-class representation of design decisions, the fact that these design decisions are cross-cutting and intertwined, that these problems lead to high maintenance cost, because of which design rules and constraints are easily violated and obsolete design decisions are not removed." (Jan Bosch, "Software architecture: The next step", 2004)

"As a noun, design is the named (although sometimes unnamable) structure or behavior of a system whose presence resolves or contributes to the resolution of a force or forces on that system. A design thus represents one point in a potential decision space. A design may be singular (representing a leaf decision) or it may be collective (representing a set of other decisions). As a verb, design is the activity of making such decisions. Given a large set of forces, a relatively malleable set of materials, and a large landscape upon which to play, the resulting decision space may be large and complex. As such, there is a science associated with design (empirical analysis can point us to optimal regions or exact points in this design space) as well as an art (within the degrees of freedom that range beyond an empirical decision; there are opportunities for elegance, beauty, simplicity, novelty, and cleverness). All architecture is design but not all design is architecture. Architecture represents the significant design decisions that shape a system, where significant is measured by cost of change." (Grady Booch, "On design", 2006)

"The goal for our software architecture is to provide the key mechanisms that are required to implement a wide variety of cross-layer adaptations described by our taxonomy. Our strategy for developing such an architecture is actually to create two architectures, a 'conceptual' one, followed by a 'concrete' one." (Soon H Choi, "A Software Architecture for Cross-layer Wireless Networks", 2008)

"A good system design is based on a sound conceptual model (architecture). A system design that has no conceptual structure and little logic to its organization is ultimately going to be unsuccessful. Good architecture will address all the requirements of the system at the right level of abstraction." (Vasudeva Varma, "Software Architecture: A Case Based Approach", 2009)

"Architecting is both an art and a science - both synthesis and analysis, induction and deduction, and conceptualization and certification - using guidelines from its art and methods from its science. As a process, it is distinguished from systems engineering in its greater use of heuristic reasoning, lesser use of analytics, closer ties to the client, and particular concern with certification of readiness for use."  (Mark W Maier, "The Art Systems of Architecting" 3rd Ed., 2009)

"Architecting is creating and building structures - that is, 'structuring'. Systems architecting is creating and building systems. It strives for fit, balance, and compromise among the tensions of client needs and resources, technology, and multiple stakeholder interests."  (Mark W Maier, "The Art Systems of Architecting" 3rd Ed., 2009)

"Taking a systems approach means paying close attention to results, the reasons we build a system. Architecture must be grounded in the client’s/user’s/customer’s purpose. Architecture is not just about the structure of components. One of the essential distinguishing features of architectural design versus other sorts of engineering design is the degree to which architectural design embraces results from the perspective of the client/user/customer. The architect does not assume some particular problem formulation, as “requirements” is fixed. The architect engages in joint exploration, ideally directly with the client/user/customer, of what system attributes will yield results worth paying for."  (Mark W Maier, "The Art Systems of Architecting" 3rd Ed., 2009)

"An architecture is the response to the integrated collections of models and views within the problem area being examined." (Charles D Tupper, "Data Architecture: From Zen to Reality", 2011)

"An architecture represents combined perspectives in a structured format that is easily viewable and explains the context of the area being analyzed to all those viewing it." (Charles D Tupper, "Data Architecture: From Zen to Reality", 2011)

"Using architecture leads to foundational stability, not rigidity. As long as the appropriate characteristics are in place to ensure positive architectural evolution, the architecture will remain a living construct. Well-developed architectures are frameworks that evolve as the business evolves." (Charles D Tupper, "Data Architecture: From Zen to Reality", 2011)

"A software architecture encompasses the significant decisions about the organization of the software system, the selection of structural elements and interfaces by which the system is composed, and determines their behavior through collaboration among these elements and their composition into progressively larger subsystems. Hence, the software architecture provides the skeleton of a system around which all other aspects of a system revolve." (Muhammad A Babar et al, "Agile Software Architecture Aligning Agile Processes and Software Architectures", 2014)

"Good architecture is all about splitting stuff reliably into self-contained parcels that allow work on them to continue relatively independently in parallel (often these days in different locations)." (Richard Hopkins & Stephen Harcombe, "Agile Architecting: Enabling the Delivery of Complex Agile Systems Development Projects", 2014)

"Good architecture provides good interfaces that separate the shear layers of its implementation: a necessity for evolution and maintenance. Class-oriented programming puts both data evolution and method evolution in the same shear layer: the class. Data tend to remain fairly stable over time, while methods change regularly to support new services and system operations. The tension in these rates of change stresses the design." (James O Coplien & Trygve Reenskaug, "The DCI Paradigm: Taking Object Orientation into the Architecture World", 2014)

"In more ways than one, architecture is all about avoiding bottlenecks. In architecture, the term bottleneck typically refers to a design problem that is preventing processing from occurring at full speed. [...] A good architecture will avoid bottlenecks in both." (Richard Hopkins & Stephen Harcombe, "Agile Architecting: Enabling the Delivery of Complex Agile Systems Development Projects", 2014)

"There is a tendency to believe that good architecture leads to systems that perform better and are more secure, but such claims relate less to any given architectural principle than to the timing of big-picture deliberations in the design cycle and to the proper engagement of suitable stakeholders." (James O Coplien & Trygve Reenskaug, "The DCI Paradigm: Taking Object Orientation into the Architecture World", 2014)

"Any software project must have a technical leader, who is responsible for all technical decisions made by the team and have enough authority to make them. Responsibility and authority are two mandatory components that must be present in order to make it possible to call such a person an architect." (Yegor Bugayenko, "Code Ahead", 2018)

"Just by making the architect role explicit, a team can effectively resolve many technical conflicts." (Yegor Bugayenko, "Code Ahead", 2018)

"To make technical decisions, a result-oriented team needs a strong architect and a decision making process, not meetings." (Yegor Bugayenko, "Code Ahead", 2018)

"[...] an architect’s work [...] comprises the set of strategic and technical models that create a context for position (capabilities), velocity (directedness, ability to adjust), and potential (relations) to harmonize strategic business and technology goals." (Eben Hewitt, "Technology Strategy Patterns: Architecture as strategy" 2nd Ed., 2019)

"Software architecture is the process and product of creating technical systems designs. Architecture may include a specification of resources, patterns, conventions, and communication protocols, among other details." (Morgan Evans, "Engineering Manager's Handbook", 2023)

"Systems architecture is the portion of any project over which the engineering team has the most control. This design is usually less of a collaboration between different functions and more clearly in the domain of engineers. As such, engineering managers have a high responsibility to own this process and its decisions. To produce the best decisions possible, you must have the right decision-building blocks: complete information to work from and a structured methodology to guide you." (Morgan Evans, "Engineering Manager's Handbook", 2023)

"Architecture begins where engineering ends." (Walter Gropius, [speech])

"Architecture is the tension between coupling and cohesion." (Neal Ford)

"Programming without an overall architecture or design in mind is like exploring a cave with only a flashlight: You don't know where you've been, you don't know where you're going, and you don't know quite where you are." (Danny Thorpe)

"The fundamental organization of a system embodied in its components, their relationships to each other, and to the environment, and the principles guiding its design and evolution." (ANSI/IEEE Std 1471: 2000)

🏗️Software Engineering: Programmers (Just the Quotes)

"Programmers should never be satisfied with languages which permit them to program everything, but to program nothing of interest easily." (Alan Perlis, "The Synthesis of Algorithmic Systems", 1966)

"The competent programmer is fully aware of the strictly limited size of his own skull; therefore he approaches the programming task in full humility, and among other things he avoids clever tricks like the plague." (Edsger W Dijkstra, "The Humble Programmer", 1972) 

"The effective exploitation of his powers of abstraction must be regarded as one of the most vital activities of a competent programmer." (Edsger W Dijkstra, "The Humble Programmer", 1972)

"The beginning of wisdom for a programmer is to recognize the difference between getting his program to work and getting it right. A program which does not work is undoubtedly wrong; but a program which does work is not necessarily right. It may still be wrong because it is hard to understand; or because it is hard to maintain as the problem requirements change; or because its structure is different from the structure of the problem; or because we cannot be sure that it does indeed work." (Michael A Jackson, "Principles of Program Design", 1975)

"The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air, from air, creating by exertion of the imagination. Few media of creation are so flexible, so easy to polish and rework, so readily capable of realizing grand conceptual structures. […] Yet the program construct, unlike the poet's words, is real in the sense that it moves and works, producing visible outputs separate from the construct itself. […] The magic of myth and legend has come true in our time. One types the correct incantation on a keyboard, and a display screen comes to life, showing things that never were nor could be." (Fred Brooks, The Mythical Man-Month: Essays, 1975) 

"There is no programming language, no matter how structured, that will prevent programmers from making bad programs. (Larry Flon, "On research in structured programming". SIGPLAN Not. 10(10), 1975)


"The computer programmer is a creator of universes for which he alone is the lawgiver. No playwright, no stage director, no emperor, however powerful, has ever exercised such absolute authority to arrange a stage or field of battle and to command such unswervingly dutiful actors or troops." (Joseph Weizenbaum, "Computer Power and Human Reason", 1976)

"Any fool can write code that a computer can understand. Good programmers write code that humans can understand." (Martin Fowler, "Refactoring: Improving the Design of Existing Code", 1999)

"Good programmers know what to write. Great ones know what to rewrite." (Eric S Raymond, "The Cathedral and the Bazaar", 1999)

"Computer programming is tremendous fun. Like music, it is a skill that derives from an unknown blend of innate talent and constant practice. Like drawing, it can be shaped to a variety of ends – commercial, artistic, and pure entertainment. Programmers have a well-deserved reputation for working long hours, but are rarely credited with being driven by creative fevers. Programmers talk about software development on weekends, vacations, and over meals not because they lack imagination, but because their imagination reveals worlds that others cannot see." (Larry O'Brien & Bruce Eckel, "Thinking in C#", 2003)

"One of the worst symptoms of a dysfunctional team is when each programmer builds a wall around his code and refuses to let other programmers touch it." (Robert C Martin,"The Clean Coder: A code of conduct for professional programmers", 2011)

"It is not loyalty or internal motivation that drives us programmers forward. We must write our code when the road to our personal success is absolutely clear for us and writing high quality code obviously helps us move forward on this road. To make this happen, the management has to define the rules of the game, also known as "process", and make sure they are strictly enforced, which is much more difficult than 'being agile'." (Yegor Bugayenko, "Code Ahead", 2018)

"Automated testing is a safety net that protects the program from its programmers." (Yegor Bugayenko, "Code Ahead", 2018)

"Quality is a product of a conflict between programmers and testers." (Yegor Bugayenko, "Code Ahead", 2018)

"Quality must be enforced, otherwise it won't happen. We programmers must be required to write tests, otherwise we won't do it." (Yegor Bugayenko, "Code Ahead", 2018)

"We must not blame programmers for their bugs. They belong to them only until the code is merged to the repository. After that, all bugs are ours!" (Yegor Bugayenko, "Code Ahead", 2018)

"We, newbies and young programmers, don't like chaos because it makes us dependent on experts. We have to beg for information and feel bad." (Yegor Bugayenko, "Code Ahead", 2018)

"The environment that nutures creative programmers kills management and marketing types - and vice versa." (Orson S Card, "How Software Companies Die")
Related Posts Plugin for WordPress, Blogger...

About Me

My photo
Koeln, NRW, Germany
IT Professional with more than 24 years experience in IT in the area of full life-cycle of Web/Desktop/Database Applications Development, Software Engineering, Consultancy, Data Management, Data Quality, Data Migrations, Reporting, ERP implementations & support, Team/Project/IT Management, etc.