💠SQL Server: Using the R Language in SQL Server 2016 (Hello World & Working with Data Frames) [new feature]

One of the most interesting features coming with SQL Server 2016 is the possibility to run external scripts written in the R language or Python, taking thus advantage of the numerical and statistical packages coming with the respective languages. The next examples are based on the R language.

As the scripts in R are considered as external scripts, is needed first to enable the 'external scripts enabled' configuration option by using the following script (a server restart is required):

-- enable external scripts 
sp_configure 'external scripts enabled', 1;
RECONFIGURE WITH OVERRIDE;

To makes sure that the functionality works as expected, it makes sense to attempt first a "hello world" example:

-- hello world script
EXECUTE sp_execute_external_script 

        @language = N'R',  
        @script = N'print("Hello world")'

The R language come with a few predefined datasets and for the following examples I’ll be using the mtcars dataset. Because the dataset contains several columns, I will use only the first 3. For this one care run a script as the following into the R console:

Usually it’s useful to look first at the structure of the dataset, this by using str(mtcars) command in the R console:

To return the dataset from R one can use the following call to the sp_execute_external_script stored procedure:

  -- returning the first 3 columns 
 EXEC sp_execute_external_script  
       @language = N'R'  
     , @script = N'cars <- mtcars[1:3];'
     , @input_data_1 = N''  
     , @output_data_1_name = N'cars'
     WITH RESULT SETS (("mpg" float not null 
         , "cyl" float not null 
         , "disp" float not null 
       ));  

As can be seen, besides the script is needed to define a variable in which the returning dataset is stored, as well the resulting dataset. Unfortunately, this script doesn’t return rows’ names. To do that I had to use a small trick by concatenating the initial data frame with the one resulting from row’s names. (Please let me know if you have another way of achieving the same.)

The script becomes:

 -- returning the first 3 columns including rows' name
 EXEC   sp_execute_external_script  
       @language = N'R'  
     , @script = N'cars <- data.frame(rownames(mtcars), mtcars[1:3]);'
     , @input_data_1 = N''  
     , @output_data_1_name = N'cars'
     WITH RESULT SETS (("Car" varchar(100)
      , "mpg" float not null 
      , "cyl" float not null 
      , "disp" float not null 
       ));  

To reuse the script, it can be included in a stored procedure, similarly like the examples provided by the Microsoft documentation for the sp_execute_external_script stored procedure.

A few pointers:
1. It’s useful to test your scripts first in the R console.
2. If 'external scripts enabled' was not enabled, then the following error message will appear:
Msg 39023, Level 16, State 1, Procedure sp_execute_external_script, Line 1 [Batch Start Line 22]
'sp_execute_external_script' is disabled on this instance of SQL Server. Use sp_configure 'external scripts enabled' to enable it.
3. It might be needed to start the “SQL Server Launchpad” service manually, a hint in this direction comes from the following error message:
Msg 39011, Level 16, State 1, Line 24
SQL Server was unable to communicate with the LaunchPad service. Please verify the configuration of the service. 
4. Once the examples tested, it might be recommended to disable the 'external scripts enabled' configuration option as long is not needed anymore.
5. Check the various ways to analyse the mtcars dataset using the R language: https://rpubs.com/BillB/217355

📊R Language: Drawing Function Plots (Part I - Basic Curves)

Besides the fact that is free, one of the advantages of the R language is that it allows drawing function plots with simple commands. All one needs is software package like Microsoft R Open and one is set to go.

For example, to draw the function f(x) = x^3-3*x+2 all one needs to do is to type the following command into the console:

curve(x^3-3*x+2, -3,3)

One can display a second function into the same chart by adding a second command and repeat the process further as needed:

curve(x^2-3*x+2, add=TRUE, col="blue")
curve(x^4-3*x+2, add=TRUE, col="red")
curve(x^5-3*x+2, add=TRUE, col="green")

As one can see a pattern emerges already …

One could easily display the plots in their one section by defining an array on the display device, allowing thus to focus on special characteristics of the functions:

par(mfrow=c(2,2))
curve(x^3-3*x+2, -3,3)
curve(x^2-3*x+2, -3,3, col="blue")
curve(x^4-3*x+2, -3,3, col="red")
curve(x^5-3*x+2, -3,3, col="green")

One can be creative and display the functions using a loop:

par(mfrow=c(2,2))
for(n in c(2:5)){
curve(x^n-3*x+2, -3,3)
}

Similarly, one can plot trigonometric functions:

par(mfrow=c(2,3))
curve(sin(x),-pi,pi) 
curve(cos(x),-pi,pi) 
curve(tan(x),-pi,pi) 
curve(1/tan(x),-pi,pi) 
curve(asin(x),-1,1)
curve(acos(x),-1,1)

The possibilities are endless. For complex functions one can include the function into an r function:

myFunction<- function(x) sin(cos(x)*exp(-x/2))
curve(myFunction, -15, 15, n=1000)

For the SQL Server developers, what’s even greater is the possibility of using Management Studio for the same, though that’s a topic for another post.

 

Happy coding!

Previous Post <<||>> Next Post

💻IT: Technology (Just the Quotes)

"Systems engineering embraces every scientific and technical concept known, including economics, management, operations, maintenance, etc. It is the job of integrating an entire problem or problem to arrive at one overall answer, and the breaking down of this answer into defined units which are selected to function compatibly to achieve the specified objectives. [...] Instrument and control engineering is but one aspect of systems engineering - a vitally important and highly publicized aspect, because the ability to create automatic controls within overall systems has made it possible to achieve objectives never before attainable, While automatic controls are vital to systems which are to be controlled, every aspect of a system is essential. Systems engineering is unbiased, it demands only what is logically required. Control engineers have been the leaders in pulling together a systems approach in the various technologies." (Instrumentation Technology, 1957)

"Doing engineering is practicing the art of the organized forcing of technological change." (George Spencer-Brown, Electronics, Vol. 32 (47),  1959)

"The decision which achieves organization objectives must be both (1) technologically sound and (2) carried out by people. If we lose sight of the second requirement or if we assume naively that people can be made to carry out whatever decisions are technically soundwe run the risk of decreasing rather than increasing the effectiveness of the organization." (Douglas McGregor, "The Human Side of Enterprise", 1960)

"Any sufficiently advanced technology is indistinguishable from magic." (Arthur C Clarke, "Profiles of the Future: An Inquiry into the Limits of the Possible", 1962)

"Science is the reduction of the bewildering diversity of unique events to manageable uniformity within one of a number of symbol systems, and technology is the art of using these symbol systems so as to control and organize unique events. Scientific observation is always a viewing of things through the refracting medium of a symbol system, and technological praxis is always handling of things in ways that some symbol system has dictated. Education in science and technology is essentially education on the symbol level." (Aldous L Huxley, "Essay", Daedalus, 1962)

"Engineering is the art of skillful approximation; the practice of gamesmanship in the highest form. In the end it is a method broad enough to tame the unknown, a means of combing disciplined judgment with intuition, courage with responsibility, and scientific competence within the practical aspects of time, of cost, and of talent. This is the exciting view of modern-day engineering that a vigorous profession can insist be the theme for education and training of its youth. It is an outlook that generates its strength and its grandeur not in the discovery of facts but in their application; not in receiving, but in giving. It is an outlook that requires many tools of science and the ability to manipulate them intelligently In the end, it is a welding of theory and practice to build an early, strong, and useful result. Except as a valuable discipline of the mind, a formal education in technology is sterile until it is applied." (Ronald B Smith, "Professional Responsibility of Engineering", Mechanical Engineering Vol. 86 (1), 1964)

"It is a commonplace of modern technology that there is a high measure of certainty that problems have solutions before there is knowledge of how they are to be solved." (John K Galbraith, "The New Industrial State", 1967)

"In many ways, project management is similar to functional or traditional management. The project manager, however, may have to accomplish his ends through the efforts of individuals who are paid and promoted by someone else in the chain of command. The pacing factor in acquiring a new plant, in building a bridge, or in developing a new product is often not technology, but management. The technology to accomplish an ad hoc project may be in hand but cannot be put to proper use because the approach to the management is inadequate and unrealistic. Too often this failure can be attributed to an attempt to fit the project to an existing management organization, rather than molding the management to fit the needs of the project. The project manager, therefore, is somewhat of a maverick in the business world. No set pattern exists by which he can operate. His philosophy of management may depart radically from traditional theory." (David I Cleland & William R King, "Systems Analysis and Project Management", 1968)

"Technological invention and innovation are the business of engineering. They are embodied in engineering change." (Daniel V DeSimone & Hardy Cross, "Education for Innovation", 1968)

"Advanced technology required the collaboration of diverse professions and organizations, often with ambiguous or highly interdependent jurisdictions. In such situations, many of our highly touted rational management techniques break down; and new non-engineering approaches are necessary for the solution of these 'systems' problems." (Leonard R Sayles &Margaret K Chandler, "Managing Large Systems: The Large-Scale Approach", 1971)

"It follows from this that man's most urgent and pre-emptive need is maximally to utilize cybernetic science and computer technology within a general systems framework, to build a meta-systemic reality which is now only dimly envisaged. Intelligent and purposeful application of rapidly developing telecommunications and teleprocessing technology should make possible a degree of worldwide value consensus heretofore unrealizable." (Richard F Ericson, "Visions of Cybernetic Organizations", 1972)

"Technology can relieve the symptoms of a problem without affecting the underlying causes. Faith in technology as the ultimate solution to all problems can thus divert our attention from the most fundamental problem - the problem of growth in a finite system." (Donella A Meadows, "The Limits to Growth", 1972)

"Modern scientific principle has been drawn from the investigation of natural laws, technology has developed from the experience of doing, and the two have been combined by means of mathematical system to form what we call engineering." (George S Emmerson, "Engineering Education: A Social History", 1973)

"The system of nature, of which man is a part, tends to be self-balancing, self-adjusting, self-cleansing. Not so with technology." (Ernst F Schumacher, "Small is Beautiful", 1973)

"Above all, innovation is not invention. It is a term of economics rather than of technology. [...] The measure of innovation is the impact on the environment. [...] To manage innovation, a manager has to be at least literate with respect to the dynamics of innovation." (Peter F Drucker, "People and Performance", 1977)

"Numeracy has two facets-reading and writing, or extracting numerical information and presenting it. The skills of data presentation may at first seem ad hoc and judgmental, a matter of style rather than of technology, but certain aspects can be formalized into explicit rules, the equivalent of elementary syntax." (Andrew Ehrenberg, "Rudiments of Numeracy", Journal of Royal Statistical Society, 1977)

"Engineering or Technology is the making of things that did not previously exist, whereas science is the discovering of things that have long existed." (David Billington, "The Tower and the Bridge: The New Art of Structural Engineering", 1983)

"No matter how high or how excellent technology may be and how much capital may be accumulated, unless the group of human beings which comprise the enterprise works together toward one unified goal, the enterprise is sure to go down the path of decline." (Takashi Ishihara, Cherry Blossoms and Robotics, 1983)

"People’s views of the world, of themselves, of their own capabilities, and of the tasks that they are asked to perform, or topics they are asked to learn, depend heavily on the conceptualizations that they bring to the task. In interacting with the environment, with others, and with the artifacts of technology, people form internal, mental models of themselves and of the things with which they are interacting. These models provide predictive and explanatory power for understanding the interaction." (Donald A Norman, "Some observations on Mental Models", 1983)

"With the changes in technological complexity, especially in information technology, the leadership task has changed. Leadership in a networked organization is a fundamentally different thing from leadership in a traditional hierarchy." (Edgar Schein, "Organizational Culture and Leadership", 1985)

"[Computer and other technical managers] must become business managers or risk landing on the technological rubbish heap." (Jim Leeke, PC Week, 1987)

"Most managers are not capable of making decisions involving complex technological matters without help - lots of it. [...] The finest technical people on the job should have a dual role: doing technical work and advising management." (Philip W Metzger, "Managing Programming People", 1987)

"People don't want to understand all the components; they just want to make it [the technology] happen." (Bernadine Nicodemus, PC Week, 1987)

"The major problems of our work are not so much technological as sociological in nature. Most managers are willing to concede the idea that they’​​​​​​ve got more people worries than technical worries. But they seldom manage that way. They manage as though technology were their principal concern. They spend their time puzzling over the most convoluted and most interesting puzzles that their people will have to solve, almost as though they themselves were going to do the work rather than manage it. […] The main reason we tend to focus on the technical rather than the human side of the work is not because it’​​​​​​s more crucial, but because it’​​​​​​s easier to do." (Tom DeMarco & Timothy Lister, "Peopleware: Productive Projects and Teams", 1987)

"Information technology can capture and process data, and expert systems can to some extent supply knowledge, enabling people to make their own decisions. As the doers become self-managing and self-controlling, hierarchy - and the slowness and bureaucracy associated with it - disappears." (Michael M Hammer, "Reengineering Work: Don't Automate, Obliterate", Magazine, 1990) [source]

"The new information technologies can be seen to drive societies toward increasingly dynamic high-energy regions further and further from thermodynamical equilibrium, characterized by decreasing specific entropy and increasingly dense free-energy flows, accessed and processed by more and more complex social, economic, and political structures." (Ervin László, "Information Technology and Social Change: An Evolutionary Systems Analysis", Behavioral Science 37, 1992)

"Ignorance of science and technology is becoming the ultimate self-indulgent luxury." (Jeremy Bernstein, "Cranks, Quarks, and the Cosmos: Writings on Science", 1993)

"Technology is nothing. What’s important is that you have a faith in people, that they’re basically good and smart, and if you give them tools, they’ll do wonderful things with them." (Steve Jobs, Rolling Stone, 1994)

"Now that knowledge is taking the place of capital as the driving force in organizations worldwide, it is all too easy to confuse data with knowledge and information technology with information." (Peter Drucker, "Managing in a Time of Great Change", 1995)

"Commonly, the threats to strategy are seen to emanate from outside a company because of changes in technology or the behavior of competitors. Although external changes can be the problem, the greater threat to strategy often comes from within. A sound strategy is undermined by a misguided view of competition, by organizational failures, and, especially, by the desire to grow." (Michael E Porter, "What is Strategy?", Harvard Business Review, 1996)

"Management is a set of processes that can keep a complicated system of people and technology running smoothly. The most important aspects of management include planning, budgeting, organizing, staffing, controlling, and problem solving. Leadership is a set of processes that creates organizations in the first place or adapts them to significantly changing circumstances. Leadership defines what the future should look like, aligns people with that vision, and inspires them to make it happen despite the obstacles." (John P Kotter, "Leading Change", 1996)

"Networks constitute the new social morphology of our societies, and the diffusion of networking logic substantially modifies the operation and outcomes in processes of production, experience, power, and culture. While the networking form of social organization has existed in other times and spaces, the new information technology paradigm provides the material basis for its pervasive expansion throughout the entire social structure." (Manuel Castells, "The Rise of the Network Society", 1996)

"Issues of quality, timeliness and change are the conditions that are forcing us to face up to the issues of enterprise architecture. The precedent of all the older disciplines known today establishes the concept of architecture as central to the ability to produce quality and timely results and to manage change in complex products. Architecture is the cornerstone for containing enterprise frustration and leveraging technology innovations to fulfill the expectations of a viable and dynamic Information Age enterprise." (John Zachman, "Enterprise Architecture: The Issue of The Century", 1997)

"The Enterprise Architecture is the explicit description of the current and desired relationships among business and management process and information technology. It describes the 'target' situation which the agency wishes to create and maintain by managing its IT portfolio." (Franklin D Raines, 1997)

"All things being equal, choose technology that connects. […] This aspect of technology has increasing importance, at times overshadowing such standbys as speed and price. If you are in doubt about what technology to purchase, get the stuff that will connect the most widely, the most often, and in the most ways. Avoid anything that resembles an island, no matter how well endowed that island is." (Kevin Kelly, "New Rules for the New Economy: 10 radical strategies for a connected world", 1998)

"Beauty is more important in computing than anywhere else in technology because software is so complicated. Beauty is the ultimate defense against complexity." (David Gelernter, "Machine Beauty: Elegance And The Heart Of Technolog", 1998)

"Modelling techniques on powerful computers allow us to simulate the behaviour of complex systems without having to understand them.  We can do with technology what we cannot do with science.  […] The rise of powerful technology is not an unconditional blessing.  We have  to deal with what we do not understand, and that demands new  ways of thinking." (Paul Cilliers,"Complexity and Postmodernism: Understanding Complex Systems", 1998)

"Technology is no panacea. It will never solve the ills or injustices of society. Technology can do only one thing for us - but it is an astonishing thing: Technology brings us an increase in opportunities." (Kevin Kelly, "New Rules for the New Economy: 10 radical strategies for a connected world", 1998)

"A primary reason that evolution - of life-forms or technology - speeds up is that it builds on its own increasing order." (Ray Kurzweil, "The Age of Spiritual Machines: When Computers Exceed Human Intelligence", 1999) 

"As systems became more varied and more complex, we find that no single methodology suffices to deal with them. This is particularly true of what may be called information intelligent systems - systems which form the core of modern technology. To conceive, design, analyze and use such systems we frequently have to employ the totality of tools that are available. Among such tools are the techniques centered on fuzzy logic, neurocomputing, evolutionary computing, probabilistic computing and related methodologies. It is this conclusion that formed the genesis of the concept of soft computing." (Lotfi A Zadeh, "The Birth and Evolution of Fuzzy Logic: A personal perspective", 1999)

"Enterprise architecture is a family of related architecture components. This include information architecture, organization and business process architecture, and information technology architecture. Each consists of architectural representations, definitions of architecture entities, their relationships, and specification of function and purpose. Enterprise architecture guides the construction and development of business organizations and business processes, and the construction and development of supporting information systems." (Gordon B Davis, "The Blackwell encyclopedic dictionary of management information systems"‎, 1999)

"Enterprise architecture is a holistic representation of all the components of the enterprise and the use of graphics and schemes are used to emphasize all parts of the enterprise, and how they are interrelated. [...] Enterprise architectures are used to deal with intra-organizational processes, interorganizational cooperation and coordination, and their shared use of information and information technologies. Business developments, such as outsourcing, partnership, alliances and Electronic Data Interchange, extend the need for architecture across company boundaries." (Gordon B Davis," The Blackwell encyclopedic dictionary of management information systems"‎, 1999)

"We do not learn much from looking at a model - we learn more from building the model and manipulating it. Just as one needs to use or observe the use of a hammer in order to really understand its function, similarly, models have to be used before they will give up their secrets. In this sense, they have the quality of a technology - the power of the model only becomes apparent in the context of its use." (Margaret Morrison & Mary S Morgan, "Models as mediating instruments", 1999)

"Periods of rapid change and high exponential growth do not, typically, last long. A new equilibrium with a new dominant technology and/or competitor is likely to be established before long. Periods of punctuation are therefore exciting and exhibit unusual uncertainty. The payoff from establishing a dominant position in this short time is therefore extraordinarily high. Dominance is more likely to come from skill in marketing and positioning than from superior technology itself." (Richar Koch, "The Power Laws", 2000)

"The business changes. The technology changes. The team changes. The team members change. The problem isn't change, per se, because change is going to happen; the problem, rather, is the inability to cope with change when it comes." (Kent Beck, "Extreme Programming Explained", 2000)

"A well-functioning team of adequate people will complete a project almost regardless of the process or technology they are asked to use (although the process and technology may help or hinder them along the way)." (Alistair Cockburn, "Agile Software Development", 2001)

"An Enterprise Architecture is a dynamic and powerful tool that helps organisations understand their own structure and the way they work. It provides a ‘map’ of the enterprise and a ‘route planner’ for business and technology change. A well-constructed Enterprise Architecture provides a foundation for the ‘Agile’ business." (Bob Jarvis, "Enterprise Architecture: Understanding the Bigger Picture - A Best Practice Guide for Decision Makers in IT", 2003)

"Normally an EA takes the form of a comprehensive set of cohesive models that describe the structure and functions of an enterprise. An important use is in systematic IT planning and architecting, and in enhanced decision-making. The EA can be regarded as the ‘master architecture’ that contains all the subarchitectures for an enterprise. The individual models in an EA are arranged in a logical manner that provides an ever-increasing level of detail about the enterprise: its objectives and goals; its processes and organisation; its systems and data; the technology used and any other relevant spheres of interest." (Bob Jarvis, "Enterprise Architecture: Understanding the Bigger Picture - A Best Practice Guide for Decision Makers in IT", 2003)

"Technology can relieve the symptoms of a problem without affecting the underlying causes. Faith in technology as the ultimate solution to all problems can thus divert our attention from the most fundamental problem - the problem of growth in a finite system - and prevent us from taking effective action to solve it." (Donella H Meadows & Dennis L Meadows, "The Limits to Growth: The 30 Year Update", 2004)

"To turn really interesting ideas and fledgling technologies into a company that can continue to innovate for years, it requires a lot of disciplines."  (Steve Jobs, BusinessWeek, 2004)

"You need a very product-oriented culture, even in a technology company. Lots of companies have tons of great engineers and smart people. But ultimately, there needs to be some gravitational force that pulls it all together. Otherwise, you can get great pieces of technology all floating around the universe." (Steve Jobs, Newsweek, 2004)

"Although the Singularity has many faces, its most important implication is this: our technology will match and then vastly exceed the refinement and suppleness of what we regard as the best of human traits." (Ray Kurzweil, "The Singularity is Near", 2005)

"The Singularity will represent the culmination of the merger of our biological thinking and existence with our technology, resulting in a world that is still human but that transcends our biological roots. There will be no distinction, post-Singularity, between human and machine or between physical and virtual reality. If you wonder what will remain unequivocally human in such a world, it’s simply this quality: ours is the species that inherently seeks to extend its physical and mental reach beyond current limitations." (Ray Kurzweil, "The Singularity is Near", 2005)

"Businesses are themselves a form of design. The design of a business encompasses its strategy, organizational structure, management processes, culture, and a host of other factors. Business designs evolve over time through a process of differentiation, selection, and amplification, with the market as the ultimate arbiter of fitness [...] the three-way coevolution of physical technologies, social technologies, and business designs [...] accounts for the patterns of change and growth we see in the economy." (Eric D Beinhocker, "The Origin of Wealth. Evolution, complexity, and the radical remaking of economics", 2006)

"Enterprise architecture is the organizing logic for business processes and IT infrastructure reflecting the integration and standardization requirements of a company's operation model. […] The key to effective enterprise architecture is to identify the processes, data, technology, and customer interfaces that take the operating model from vision to reality." (Jeanne W Ross et al, "Enterprise architecture as strategy: creating a foundation for business", 2006)

"Chance is just as real as causation; both are modes of becoming.  The way to model a random process is to enrich the mathematical theory of probability with a model of a random mechanism. In the sciences, probabilities are never made up or 'elicited' by observing the choices people make, or the bets they are willing to place.  The reason is that, in science and technology, interpreted probability exactifies objective chance, not gut feeling or intuition. No randomness, no probability." (Mario Bunge, "Chasing Reality: Strife over Realism", 2006)

"Most dashboards fail to communicate efficiently and effectively, not because of inadequate technology (at least not primarily), but because of poorly designed implementations. No matter how great the technology, a dashboard's success as a medium of communication is a product of design, a result of a display that speaks clearly and immediately. Dashboards can tap into the tremendous power of visual perception to communicate, but only if those who implement them understand visual perception and apply that understanding through design principles and practices that are aligned with the way people see and think." (Stephen Few, "Information Dashboard Design", 2006)

"The big part of the challenge is that data quality does not improve by itself or as a result of general IT advancements. Over the years, the onus of data quality improvement was placed on modern database technologies and better information systems. [...] In reality, most IT processes affect data quality negatively, Thus, if we do nothing, data quality will continuously deteriorate to the point where the data will become a huge liability." (Arkady Maydanchik, "Data Quality Assessment", 2007)

"The corporate data universe consists of numerous databases linked by countless real-time and batch data feeds. The data continuously move about and change. The databases are endlessly redesigned and upgraded, as are the programs responsible for data exchange. The typical result of this dynamic is that information systems get better, while data deteriorates. This is very unfortunate since it is the data quality that determines the intrinsic value of the data to the business and consumers. Information technology serves only as a magnifier for this intrinsic value. Thus, high quality data combined with effective technology is a great asset, but poor quality data combined with effective technology is an equally great liability." (Arkady Maydanchik, "Data Quality Assessment", 2007)

"Enterprise architecture is the process of translating business vision and strategy into effective enterprise change by creating, communicating and improving the key requirements, principles and models that describe the enterprise's future state and enable its evolution. The scope of the enterprise architecture includes the people, processes, information and technology of the enterprise, and their relationships to one another and to the external environment. Enterprise architects compose holistic solutions that address the business challenges of the enterprise and support the governance needed to implement them." (Anne Lapkin et al, "Gartner Clarifies the Definition of the Term 'Enterprise Architecture", 2008)

"Synergy occurs when organizational parts interact to produce a joint effect that is greater than the sum of the parts acting alone. As a result the organization may attain a special advantage with respect to cost, market power, technology, or employee." (Richard L Daft, "The Leadership Experience" 4th Ed., 2008)

"The butterfly effect demonstrates that complex dynamical systems are highly responsive and interconnected webs of feedback loops. It reminds us that we live in a highly interconnected world. Thus our actions within an organization can lead to a range of unpredicted responses and unexpected outcomes. This seriously calls into doubt the wisdom of believing that a major organizational change intervention will necessarily achieve its pre-planned and highly desired outcomes. Small changes in the social, technological, political, ecological or economic conditions can have major implications over time for organizations, communities, societies and even nations." (Elizabeth McMillan, "Complexity, Management and the Dynamics of Change: Challenges for practice", 2008)

"What’s next for technology and design? A lot less thinking about technology for technology’s sake, and a lot more thinking about design. Art humanizes technology and makes it understandable. Design is needed to make sense of information overload. It is why art and design will rise in importance during this century as we try to make sense of all the possibilities that digital technology now affords." (John Maeda, "Why Apple Leads the Way in Design", 2010) 

"Enterprise Architecture presently appears to be a grossly misunderstood concept among management. It is NOT an Information Technology issue. It is an ENTERPRISE issue. It is likely perceived to be an Information Technology issue as opposed to a Management issue for two reasons: (1) Awareness of it tends to surface in the Enterprise through the Information Systems community. (2) Information Technology people seem to have the skills to do Enterprise Architecture if any Enterprise Architecture is being or is to be done." (John A Zachman, 2011)

"Today, technology has lowered the barrier for others to share their opinion about what we should be focusing on. It is not just information overload; it is opinion overload." (Greg McKeown, "Essentialism: The Disciplined Pursuit of Less", 2014)

"We have let ourselves become enchanted by big data only because we exoticize technology. We’re impressed with small feats accomplished by computers alone, but we ignore big achievements from complementarity because the human contribution makes them less uncanny. Watson, Deep Blue, and ever-better machine learning algorithms are cool. But the most valuable companies in the future won’t ask what problems can be solved with computers alone. Instead, they’ll ask: how can computers help humans solve hard problems?" (Peter Thiel & Blake Masters, "Zero to One: Notes on Startups, or How to Build the Future", 2014)

"Technological change is discontinuous and difficult. It is a radical change in that it forces people to deal with the world in a different way, that is, it changes the world of experience." (William Byers, "Deep Thinking: What Mathematics Can Teach Us About the Mind", 2015)

"The problem with artificial intelligence and information technology is that they promise a methodology that would lead to a way of solving all problems - a self-generating technology that would apply to all situations without the need for new human insights and leaps of creativity." (William Byers, "Deep Thinking: What Mathematics Can Teach Us About the Mind", 2015)

"Technology systems are difficult to wrangle. Our systems grow in accidental complexity and complication over time. Sometimes we can succumb to thinking that other people really hold the cards, that they have the puppet strings we don’t." (Eben Hewitt, "Technology Strategy Patterns: Architecture as strategy" 2nd Ed., 2019)

"Technology is not a magic pill that can solve inadequacies in processes." (Jared Lane, "Why Companies Should Stop Making Digital Transformation A Science Project", 2021) [source]

"Always remember what you originally wanted the system to accomplish. Having the latest, greatest system and a flashy data center to boot is not what data processing is supposed to be all about. It is supposed to help the bottom line, not hinder it." (Richard S Rubin)

"The first rule of any technology used in a business is that automation applied to an efficient operation will magnify the efficiency. The second is that automation applied to an inefficient operation will magnify the inefficiency." (Bill Gates)

🤝Governance: Control (Just the Quotes)

"To manage is to forecast and plan, to organize, to command, to coordinate and to control. To foresee and plan means examining the future and drawing up the plan of action. To organize means building up the dual structure, material and human, of the undertaking. To command means binding together, unifying and harmonizing all activity and effort. To control means seeing that everything occurs in conformity with established rule and expressed demand." (Henri Fayol, 1916)

"The concern of OR with finding an optimum decision, policy, or design is one of its essential characteristics. It does not seek merely to define a better solution to a problem than the one in use; it seeks the best solution... [It] can be characterized as the application of scientific methods, techniques, and tools to problems involving the operations of systems so as to provide those in control of the operations with optimum solutions to the problems." (C West Churchman et al, "Introduction to Operations Research", 1957)

"Management is a distinct process consisting of planning, organising, actuating and controlling; utilising in each both science and art, and followed in order to accomplish pre-determined objectives." (George R Terry, "Principles of Management", 1960)

"The term architecture is used here to describe the attributes of a system as seen by the programmer, i.e., the conceptual structure and functional behavior, as distinct from the organization of the data flow and controls, the logical design, and the physical implementation." (Gene Amdahl et al, "Architecture of the IBM System", IBM Journal of Research and Development. Vol 8 (2), 1964)

"If cybernetics is the science of control, management is the profession of control." (Anthony S Beer, "Decision and Control", 1966)

"Most of our beliefs about complex organizations follow from one or the other of two distinct strategies. The closed-system strategy seeks certainty by incorporating only those variables positively associated with goal achievement and subjecting them to a monolithic control network. The open-system strategy shifts attention from goal achievement to survival and incorporates uncertainty by recognizing organizational interdependence with environment. A newer tradition enables us to conceive of the organization as an open system, indeterminate and faced with uncertainty, but subject to criteria of rationality and hence needing certainty." (James D Thompson, "Organizations in Action", 1967)

"Policy-making, decision-taking, and control: These are the three functions of management that have intellectual content." (Anthony S Beer, "Management Science" , 1968)

"The management of a system has to deal with the generation of the plans for the system, i. e., consideration of all of the things we have discussed, the overall goals, the environment, the utilization of resources and the components. The management sets the component goals, allocates the resources, and controls the system performance." (C West Churchman, "The Systems Approach", 1968)

"One difficulty in developing a good [accounting] control system is that quantitative results will differ according to the accounting principles used, and accounting principles may change." (Ernest Dale, "Readings in Management", 1970)

"To be productive the individual has to have control, to a substantial extent, over the speed, rhythm, and attention spans with which he is working […] While work is, therefore, best laid out as uniform, working is best organized with a considerable degree of diversity. Working requires latitude to change speed, rhythm, and attention span fairly often. It requires fairly frequent changes in operating routines as well. What is good industrial engineering for work is exceedingly poor human engineering for the worker." (Peter F Drucker, "Management: Tasks, Responsibilities, Practices", 1973)

"A mature science, with respect to the matter of errors in variables, is not one that measures its variables without error, for this is impossible. It is, rather, a science which properly manages its errors, controlling their magnitudes and correctly calculating their implications for substantive conclusions." (Otis D Duncan, "Introduction to Structural Equation Models", 1975)

"Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." (Charles Goodhart, "Problems of Monetary Management: the U.K. Experience", 1975)

"When information is centralized and controlled, those who have it are extremely influential. Since information is [usually] localized in control subsystems, these subsystems have a great deal of organization influence." (Henry L Tosi & Stephen J Carroll, "Management", 1976)

"[...] when a variety of tasks have all to be performed in cooperation, synchronization, and communication, a business needs managers and a management. Otherwise, things go out of control; plans fail to turn into action; or, worse, different parts of the plans get going at different speeds, different times, and with different objectives and goals, and the favor of the 'boss' becomes more important than performance." (Peter F Drucker, "People and Performance", 1977)

"Uncontrolled variation is the enemy of quality." (W Edwards Deming, 1980)

"The key mission of contemporary management is to transcend the old models which limited the manager's role to that of controller, expert or morale booster. These roles do not produce the desired result of aligning the goals of the employees and the corporation. [...] These older models, vestiges of a bygone era, have served their function and must be replaced with a model of the manager as a developer of human resources." (Michael Durst, "Small Systems World", 1985)

"The outcome of any professional's effort depends on the ability to control working conditions." (Joseph A Raelin, "Clash of Cultures: Managers and Professionals", 1986)

"Executives have to start understanding that they have certain legal and ethical responsibilities for information under their control." (Jim Leeke, PC Week, 1987)

"Give up control even if it means the employees have to make some mistakes." (Frank Flores, Hispanic Business, 1987)

"In complex situations, we may rely too heavily on planning and forecasting and underestimate the importance of random factors in the environment. That reliance can also lead to delusions of control." (Hillel J Einhorn & Robin M. Hogarth, Harvard Business Review, 1987)

"Managers exist to plan, direct and control the project. Part of the way they control is to listen to and weigh advice. Once a decision is made, that's the way things should proceed until a new decision is reached. Erosion of management decisions by [support] people who always 'know better' undermines managers' credibility and can bring a project to grief." (Philip W Metzger, "Managing Programming People", 1987)

"To be effective, a manager must accept a decreasing degree of direct control." (Eric G Flamholtz & Yvonne Randal, "The Inner Game of Management", 1987)

"[Well-managed modern organizations] treat everyone as a source of creative input. What's most interesting is that they cannot be described as either democratically or autocratically managed. Their managers define the boundaries, and their people figure out the best way to do the job within those boundaries. The management style is an astonishing combination of direction and empowerment. They give up tight control in order to gain control over what counts: results." (Robert H Waterman, "The Renewal Factor", 1987)

"We have created trouble for ourselves in organizations by confusing control with order. This is no surprise, given that for most of its written history, leadership has been defined in terms of its control functions." (Margaret J Wheatley, "Leadership and the New Science: Discovering Order in a Chaotic World", 1992)

"Management is not founded on observation and experiment, but on a drive towards a set of outcomes. These aims are not altogether explicit; at one extreme they may amount to no more than an intention to preserve the status quo, at the other extreme they may embody an obsessional demand for power, profit or prestige. But the scientist's quest for insight, for understanding, for wanting to know what makes the system tick, rarely figures in the manager's motivation. Secondly, and therefore, management is not, even in intention, separable from its own intentions and desires: its policies express them. Thirdly, management is not normally aware of the conventional nature of its intellectual processes and control procedures. It is accustomed to confuse its conventions for recording information with truths-about-the-business, its subjective institutional languages for discussing the business with an objective language of fact and its models of reality with reality itself." (Stanford Beer, "Decision and Control", 1994)

"Without some element of governance from the top, bottom-up control will freeze when options are many. Without some element of leadership, the many at the bottom will be paralysed with choices." (Kevin Kelly, "Out of Control: The New Biology of Machines, Social Systems and the Economic World", 1995)

"Management is a set of processes that can keep a complicated system of people and technology running smoothly. The most important aspects of management include planning, budgeting, organizing, staffing, controlling, and problem solving." (John P Kotter, "Leading Change", 1996) 

"The manager [...] is understood as one who observes the causal structure of an organization in order to be able to control it [...] This is taken to mean that the manager can choose the goals of the organization and design the systems or actions to realize those goals [...]. The possibility of so choosing goals and strategies relies on the predictability provided by the efficient and formative causal structure of the organization, as does the possibility of managers staying 'in control' of their organization's development. According to this perspective, organizations become what they are because of the choices made by their managers." (Ralph D Stacey et al, "Complexity and Management: Fad or Radical Challenge to Systems Thinking?", 2000)

"Success or failure of a project depends upon the ability of key personnel to have sufficient data for decision-making. Project management is often considered to be both an art and a science. It is an art because of the strong need for interpersonal skills, and the project planning and control forms attempt to convert part of the 'art' into a science." (Harold Kerzner, "Strategic Planning for Project Management using a Project Management Maturity Model", 2001)

"The premise here is that the hierarchy lines on the chart are also the only communication conduit. Information can flow only along the lines. [...] The hierarchy lines are paths of authority. When communication happens only over the hierarchy lines, that's a priori evidence that the managers are trying to hold on to all control. This is not only inefficient but an insult to the people underneath." (Tom DeMarco, "Slack: Getting Past Burnout, Busywork, and the Myth of Total Efficiency", 2001)

"Management can be defined as the attainment of organizational goals in an effective and efficient manner through planning, organizing, staffing, directing, and controlling organizational resources." (Richard L Daft, "The Leadership Experience" 4th Ed., 2008)

"In a complex society, individuals, organizations, and states require a high degree of confidence - even if it is misplaced - in the short-term future and a reasonable degree of confidence about the longer term. In its absence they could not commit themselves to decisions, investments, and policies. Like nudging the frame of a pinball machine to influence the path of the ball, we cope with the dilemma of uncertainty by doing what we can to make our expectations of the future self-fulfilling. We seek to control the social and physical worlds not only to make them more predictable but to reduce the likelihood of disruptive and damaging shocks (e.g., floods, epidemics, stock market crashes, foreign attacks). Our fallback strategy is denial." (Richard N Lebow, "Forbidden Fruit: Counterfactuals and International Relations", 2010)

"Almost by definition, one is rarely privileged to 'control' a disaster. Yet the activity somewhat loosely referred to by this term is a substantial portion of Management, perhaps the most important part. […] It is the business of a good Manager to ensure, by taking timely action in the real world, that scenarios of disaster remain securely in the realm of Fantasy." (John Gall, "The Systems Bible: The Beginner's Guide to Systems Large and Small"[Systematics 3rd Ed.], 2011)

"Without precise predictability, control is impotent and almost meaningless. In other words, the lesser the predictability, the harder the entity or system is to control, and vice versa. If our universe actually operated on linear causality, with no surprises, uncertainty, or abrupt changes, all future events would be absolutely predictable in a sort of waveless orderliness." (Lawrence K Samuels, "Defense of Chaos", 2013)

"The problem of complexity is at the heart of mankind’s inability to predict future events with any accuracy. Complexity science has demonstrated that the more factors found within a complex system, the more chances of unpredictable behavior. And without predictability, any meaningful control is nearly impossible. Obviously, this means that you cannot control what you cannot predict. The ability ever to predict long-term events is a pipedream. Mankind has little to do with changing climate; complexity does." (Lawrence K Samuels, "The Real Science Behind Changing Climate", LewRockwell.com, August 1, 2014) 

🧭Business Intelligence: Perspectives (Part V: Data Soup - From BI to Analytics)

Business Intelligence Series

The days when everything was reduced to simple terminology like reports or queries are gone. One can see it in the market trends related to reporting or data, as well in the jargon soup the IT people use on the daily basis – Business Intelligence (BI), Data Mining (DM), Analytics, Data Science, Data Warehousing (DW), Machine Learning (ML), Artificial Intelligence (AI) and so on. What’s more confusing for the users and other spectators is the easiness with which all these concepts are used, sometimes interchangeably, and often it feels like nothing makes sense.

BI is used nowadays to refer to the technologies, architectures, methodologies, processes and practices used to transform data into what is desired as meaningful and useful information.  From its early beginnings in the 60s, the intelligence from Business Intelligence (BI) refers to the ability to apprehend the interrelationships of the facts to be processed (aka data) in such a way as to guide action towards a desired goal.

The main purpose of BI was and is to guide actions and provide a solid basis for decision making, aspect not necessarily reflected in the way organizations use their BI infrastructure. Except basic operational/tactical/strategic reports and metrics that reflect to a higher or lower degree organizations’ goals, BI often fails to provide the expected value. The causes are multiple ranging from an organizations maturity in devising a strategy and dividing it into SMART goals and objectives, to the misuse of technologies for the wrong purposes.

Despite the basic data analysis techniques, the rich visualizations and navigation functionality, BI fails often to deliver by itself more than ordinary and already known information. Information becomes valuable when it brings novelty, when it can be easily transformed into knowledge, or even better, when knowledge is extracted directly. To address the limitations of the BI a series of techniques appeared in parallel and coined in the 90s as Data Mining.

Mining is the process of obtaining something valuable from a resource. What DM tries to achieve as process is the extraction of knowledge in form patterns from the data by categorizing, clustering, identifying dependencies or anomalies. When compared with data analysis, the main characteristics of DM is the fact that is used to test models and hypotheses, and that it uses a set of semiautomatic and automatic out-of-the-box statistics packages, AI or predictive algorithms with applicability in different areas – Web,  text, speech, business processes, etc.

DM proved to be useful by allowing to build models rooted in historical data, models which allowed predicting outcome or behavior, however the models are pretty basic and there’s always a threshold beyond which they can’t go. Furthermore, the costs of preparing the data and of the needed infrastructure seem to be high compared with the benefits data mining provides. There are scenarios in which DM proves to bring benefit, while in others it raises more challenges than can solve. Privacy, security, misuse of information and the blind use of techniques without understanding the data or the models behind, are just some of such challenges.  

Information seems too common, while knowledge can become expensive to obtain. The middle way between the two found its future into another buzzword – analytics – the systematic analysis of data or statistics using specific mathematical methods. Analytics combine the agility of data analysis techniques with the power of predictive and prescriptive techniques used in DM in discovering patterns into the data. Analytics attempts to identify why it happens by using a chain of inferences resulted from data’s analyzing and understanding. From another perspective analytics seems to be a rebranded and slightly enhanced version of BI.

🔦Process Management: Business Process (Definitions)

"A business process is a collection of activities that takes one or more kinds of input and creates an output that is of value to the customer. A business process has a goal and is affected by events occurring in the external world or in other processes." (James A Champy & Michael M Hammer, "Reengineering the Corporation", 1993)

"A process is a set of linked activities that take an input and transform it to create an output. Ideally, the transformation that occurs in the process should add value to the input and create an output that is more useful and effective to the recipient either upstream or downstream."

(Henry J Johansson, "Business process reengineering: Breakpoint strategies for market dominance", 1993)

"Major operational activities or processes supported by a source system, such as orders, from which data can be collected for the analytic purposes of the data warehouse. Choosing the business process is the first of four key steps in the design of a dimensional model." (Ralph Kimball & Margy Ross, "The Data Warehouse Toolkit" 2nd Ed., 2002)

"The sequence of activities 'enclosing' the production process. These activities are common to all types of products and services, and include defining the job, negotiation with the customer, and reporting project status." (Richard D Stutzke, "Estimating Software-Intensive Systems: Projects, Products, and Processes", 2005)

"The subject areas of a business. The method by which a business is divided up. In a data warehouse, the subject areas become the fact tables." (Gavin Powell, "Beginning Database Design", 2006)

"A structured description of the activities or tasks that have to be done to fulfill a certain business need. The activities or tasks might be manual steps (human interaction) or automated steps (IT steps)." (Nicolai M Josuttis, "SOA in Practice", 2007)

"A structured and measured, managed, and controlled set of interrelated and interacting activities that uses resources to transform inputs into specified outputs." (Nathalíe Galeano, "Competency Concept in VO Breeding Environment", 2008) 

"The codification of rules and practices that constitute a business." (Judith Hurwitz et al, "Service Oriented Architecture For Dummies" 2nd Ed., 2009)

"The defined method for a range of activities that organizations perform. A business process can include anything from the steps needed to make a product to how a supply is ordered or how an invoice is created." (Tony Fisher, "The Data Asset", 2009)

"A structured description of the activities or tasks that have to be done to fulfill a certain business need. The activities or tasks might be manual steps (human interaction) or automated steps (IT steps)." (David Lyle & John G Schmidt, "Lean Integration", 2010)

"An activity as carried out by business people, including the mechanisms involved. This is in the domain of Row Two, the Business Owner’s View. Alternatively, the architect in Row Three sees a system process which is about the data transformations involved in carrying out a business process. In either case, processes can be viewed at a high level or in atomic detail." (David C Hay, "Data Model Patterns: A Metadata Map", 2010)

"A collection of activities performed to accomplish a clearly defined goal." (Linda Volonino & Efraim Turban, "Information Technology for Management" 8th Ed., 2011)

"A collection of activities designed to produce a specific output for a particular customer or market." (International Qualifications Board for Business Analysis, "Standard glossary of terms used in Software Engineering", 2011)

"A process that is intended to contribute to the overall value of an enterprise. The complex interactions between people, applications, and technologies designed to create customer value. A process is composed of activities." (DAMA International, "The DAMA Dictionary of Data Management", 2011)

"A business process is a series of steps required to execute a function that is important to an organization. Business processes include things like taking an order or setting up an account or paying a claim. In process analysis, business processes are the focus of opportunities for improvement. Organizations usually have a set of key processes that require support from other areas, like information technology." (Laura Sebastian-Coleman, "Measuring Data Quality for Ongoing Improvement ", 2012)

 "A holistic management approach for the detection, analysis, modeling, implementation, improvement and governance of the activities within or between enterprises." (Michael Fellmann et al, "Supporting Semantic Verification of Process Models", 2012)

"An activity (or set of activities) that is managed by an organization to produce some result of value to that organization, its customers, its suppliers, and/or its partners." (Graham Witt, "Writing Effective Business Rules", 2012)

"The codification of rules and practices that constitute a business." (Marcia Kaufman et al, "Big Data For Dummies", 2013)

"A coordinated set of collaborative and transactional work activities carried out to complete work steps." (Robert F Smallwood, "Information Governance: Concepts, Strategies, and Best Practices", 2014)

"The defined method for a range of activities that organizations perform. A business process can include anything from the steps needed to make a product to how a supply is ordered or how a decision is made." (Jim Davis & Aiman Zeid, "Business Transformation", 2014)

"A set of activities that teams within an organization carry out to accomplish a specific goal." (David K Pham, "From Business Strategy to Information Technology Roadmap", 2016)

"The business activities executed to deliver products or services to external customers. Business process is supported by and consumes IT-services to achieve their objectives." (by Brian Johnson & Leon-Paul de Rouw, "Collaborative Business Design", 2017)

"At its most generic, any set of activities performed by a business that is initiated by an event, transforms information, materials or business commitments, and produces an output. Value chains and large-scale business processes produce outputs that are valued by customers. Other processes generate outputs that are valued by other processes." (Appian)

🛡️Information Security: Firewall (Definitions)

"A device or program that blocks outsiders from accessing a computer connected to the Internet. Some firewalls also monitor data traffic outbound from a computer or network." (Andy Walker, "Absolute Beginner’s Guide To: Security, Spam, Spyware & Viruses", 2005)

"Software or devices that examine network traffic so that it may restrict access to network resources to unauthorized users." (Tom Petrocelli, "Data Protection and Information Lifecycle Management", 2005)

"A network security system used to monitor and restrict external and internal traffic." (Robert McCrie, "Security Operations Management" 2nd Ed., 2006)

"A firewall is part of a computer network or system that is designed to block unauthorized access over communications lines." (Michael Coles & Rodney Landrum, , "Expert SQL Server 2008 Encryption", 2008)

"A system level networking filter that restricts access based on, among other things, IP address. Firewalls form a part of an effective network security strategy. See Firewalls." (MongoDb, "Glossary", 2008)

"A piece of software that filters incoming and outgoing network traffic and stops messages that violate the rules that define allowable traffic." (Jan L Harrington, "Relational Database Design and Implementation" 3rd Ed., 2009)

"A computer system placed between the Internet and an internal subnet of an enterprise to prevent unauthorized outsiders from accessing internal data." (Paulraj Ponniah, "Data Warehousing Fundamentals for IT Professionals", 2010)

"A combination of specialized hardware and software set up to monitor traffic between an internal network and an external network (i.e. the Internet). Its primary purpose if for security and is designed to keep unauthorized outsiders from tampering with or accessing information on a networked computer system." (DAMA International, "The DAMA Dictionary of Data Management", 2011)

"Hardware and software that blocks outsiders from accessing your data and creates a secure environment for your data while permitting those with authorization, such as employees, to access information as needed." (Gina Abudi & Brandon Toropov, "The Complete Idiot's Guide to Best Practices for Small Business", 2011)

"System or group of systems that enforces an access-control policy between two networks." (Linda Volonino & Efraim Turban, "Information Technology for Management" 8th Ed., 2011)

"A device that is used to control access between two networks. Typically used when connecting a private network to the Internet as a way of protecting and securing the internal network from threats, hackers, and others. Also used when connecting two private networks (e.g., supplies, partners, etc.)." (Bill Holtsnider & Brian D Jaffe, "IT Manager's Handbook" 3rd Ed., 2012)

"A network access control system that uses rules to block or allow connections and data transmission between a private network and an untrusted network, such as the Internet." (Mark Rhodes-Ousley, "Information Security: The Complete Reference" 2nd Ed., 2013)

"A form of protection that allows one network to connect to another network while maintaining some amount of protection." ( Manish Agrawal, "Information Security and IT Risk Management", 2014)

"Software or hardware designed to control traffic. A network-based firewall is typically hardware, and it controls traffic in and out of a network. A host-based firewall is software installed on individual systems and it controls traffic in and out of individual systems." (Darril Gibson, "Effective Help Desk Specialist Skills", 2014)

"A a network security measure designed to filter out undesirable network traffic." (Weiss, "Auditing IT Infrastructures for Compliance" 2nd Ed., 2015)

"A gateway machine and its software that protects a network by filtering the traffic it allows" (Nell Dale & John Lewis, "Computer Science Illuminated" 6th Ed., 2015)

"A security barrier on your computer or network that controls what traffic is allowed to pass through." (Faithe Wempen, "Computing Fundamentals: Introduction to Computers", 2015)

"Software that blocks hackers from accessing a computer by closing unnecessary services and ports." (Faithe Wempen, "Computing Fundamentals: Introduction to Computers", 2015)

"A network device designed to selectively block unauthorized access while permitting authorized communication to devices within a subnetwork." (O Sami Saydjari, "Engineering Trustworthy Systems: Get Cybersecurity Design Right the First Time", 2018)

🛡️Information Security: Data Leak/Loss Prevention (Definitions)

"Attempts to prevent the loss of confidentiality of sensitive information by limiting the use of confidential information only for authorized purposes." (David G Hill, "Data Protection: Governance, Risk Management, and Compliance", 2009)

"A feature that protects data on laptops by enabling file-level authentication and secure erase options in the event that a laptop is lost or stolen." (CommVault, "Documentation 11.20", 2018)

"A set of technologies and inspection techniques used to classify information content contained within an object—such as a file, an email, a packet, an application or a data store - while at rest (in storage), in use (during an operation), or in transit (across a network). DLP tools also have the ability to dynamically apply a policy—such as log, report, classify, relocate, tag, and encrypt - and/or apply enterprise data rights management protections." (William Stallings, "Effective Cybersecurity: A Guide to Using Best Practices and Standards", 2018)

"The actions that organizations take to prevent unauthorized external parties from gaining access to sensitive data." (Shon Harris & Fernando Maymi, "CISSP All-in-One Exam Guide" 8th Ed., 2018)

"Data loss prevention (DLP; also known as data leak prevention) is a computer security term referring to systems that identify, monitor, and protect data in use (e.g. endpoint actions), data in motion (e.g. network actions), and data at rest (e.g. data storage) through deep content inspection, contextual security analysis of transaction (attributes of originator, data object, medium, timing, recipient/destination, and so on) and with a centralized management framework. Systems are designed to detect and prevent unauthorized use and transmission of confidential information." (Robert F Smallwood, "Information Governance for Healthcare Professionals", 2018)

 "A capability that detects and prevents violations to corporate policies regarding the use, storage, and transmission of sensitive data. Its purpose is to enforce policies to prevent unwanted dissemination of sensitive information." (Forrester)

"A systems ability to identify, monitor, and protect data in use (e.g. endpoint actions), data in motion (e.g. network actions), and data at rest (e.g. data storage) through deep packet content inspection, contextual security analysis of transaction (attributes of originator, data object, medium, timing, recipient/destination, etc.), within a centralized management framework. Data loss prevention capabilities are designed to detect and prevent the unauthorized use and transmission of NSS information." (CNSSI 4009-2015 CNSSI 1011)

"Data loss protection (DLP) describes a set of technologies and inspection techniques used to classify information content contained within an object - such as a file, email, packet, application or data store - while at rest (in storage), in use (during an operation) or in transit (across a network). DLP tools are also have the ability to dynamically apply a policy - such as log, report, classify, relocate, tag and encrypt - and/or apply enterprise data rights management protections." (Gartner)

"Data loss prevention (DLP) is a strategy for making sure that end users do not send sensitive or critical information outside the corporate network. The term is also used to describe software products that help a network administrator control what data end users can transfer." (TechTarget) [source]

"Data loss prevention (DLP) makes sure that users do not send sensitive or critical information outside the corporate network. The term describes software products that help a network administrator control the data that users can transfer." (proofpoint) [source]

🛡️Information Security: Data Breach (Definitions)

[data loss:] "Deprivation of something useful or valuable about a set of data, such as unplanned physical destruction of data or failure to preserve the confidentiality of data." (David G Hill, "Data Protection: Governance, Risk Management, and Compliance", 2009)

"The unauthorized disclosure of confidential information, notably that of identifying information about individuals." (David G Hill, "Data Protection: Governance, Risk Management, and Compliance", 2009)

"A failure of an obligation to protect against the release of secure data." (Janice M Roehl-Anderson, "IT Best Practices for Financial Managers", 2010)

"The release of secure information to an untrusted environment. Other terms for this occurrence include unintentional information disclosure, data leak, and data spill." (Craig S Mullins, "Database Administration", 2012)

"The unauthorized movement or disclosure of sensitive information to a party, usually outside the organization, that is not authorized to have or see the information." (Olivera Injac & Ramo Šendelj, "National Security Policy and Strategy and Cyber Security Risks", 2016)

"An incident in which sensitive, protected or confidential data has been viewed, stolen or used by an unauthorized body." (Güney Gürsel, "Patient Privacy and Security in E-Health", 2017)

[data leakage:] "The advertent or inadvertent sharing of private and/or confidential information." (Shalin Hai-Jew, "Beware!: A Multimodal Analysis of Cautionary Tales in Strategic Cybersecurity Messaging Online", 2018)

"A security incident involving unauthorized access to data." (Boaventura DaCosta & Soonhwa Seok, "Cybercrime in Online Gaming", 2020)

"An incident where information is accessed without authorization." (Nathan J Rodriguez, "Internet Privacy", 2020)

"A process where large amounts of private data, mostly about individuals, becomes illegally available to people who should not have access to the information." (Ananda Mitra & Yasmine Khosrowshahi, "The 2018 Facebook Data Controversy and Technological Alienation", 2021)

"This refers to any intentional or unintentional leak of secure or private or confidential data to any untrusted system. This is also referred to as information disclosure or data spill." (Srinivasan Vaidyanathan et al, "Challenges of Developing AI Applications in the Evolving Digital World and Recommendations to Mitigate Such Challenges: A Conceptual View", 2021) 

"When the information is stolen or used without consent of the system’s owner, the data stolen may cover confidential information like credit cards or passwords." (Kevser Z Meral, "Social Media Short Video-Sharing TikTok Application and Ethics: Data Privacy and Addiction Issues", 2021)

[data loss:] "The exposure of proprietary, sensitive, or classified information through either data theft or data leakage." (CNSSI 4009-2015)

🛡️Information Security: Data Privacy (Definitions)

"Right of an individual to participate in decisions regarding the collection, use, and disclosure of information personally identifiable to that individual." (Reima Suomi, "Telework and Data Privacy and Security", 2008)

"Current United States laws provide protection to private data, including students’ performance data. Online distance education environments need to address privacy issues though design of courses and security features built into record keeping systems." (Gregory C Sales, "Preparing Teachers to Teach Online", 2009)

"Personal data should not be automatically available to other persons or organizations. Even if data have been processed, each individual should be able to exercise his or her right to control access to data and related information." (Astrid Gesche, "Adapting to Virtual Third-Space Language Learning Futures", 2009)

"The right to have personally identifiable information not disclosed in any unauthorized manner." (David G Hill, "Data Protection: Governance, Risk Management, and Compliance", 2009)

"The limitation of data access to only those authorized to view the data." (DAMA International, "The DAMA Dictionary of Data Management", 2011)

"The legal, political, and ethical issues surrounding the collection and dissemination of data, the technology used, and the expectations of what information is shared with whom." (Jonathan Ferrar et al, "The Power of People: Learn How Successful Organizations Use Workforce Analytics To Improve Business Performance", 2017)

"A compliance program aimed at protection of personal information about any individual the company may poses." (Svetlana Snezhko & Ali Coskun, "Compliance in Sustainability Reporting", 2019)

"Data containing information about a person should be treated with special attention according to the organization’s data privacy policy and legislation." (Lili Aunimo et al, "Big Data Governance in Agile and Data-Driven Software Development: A Market Entry Case in the Educational Game Industry", 2019)

"The term refers to the confidentiality of information that one has and other parties are not allowed to share it without a consent of the data owner. Privacy is a measure of control for individuals about their personal information." (M Fevzi Esen & Eda Kocabas, "Personal Data Privacy and Protection in the Meeting, Incentive, Convention, and Exhibition (MICE) Industry", 2019)

"This term relates to the individual right to restrict access to their personal, health, political/philosophical views, religious affiliation and educational data. In the case of students, schools and districts have the responsibility to control access to student data, providing it is available only to those who play a role in the learning process and for a defined time span." (Beatriz Arnillas, "Tech-Savvy Is the New Street Smart: Balancing Protection and Awareness", 2019)

"Protection of personal privacy during data acquisition, storage, transmission, and usage." (Hemlata Gangwar, "Big Data Adoption: A Comparative Study of the Indian Manufacturing and Services Sectors", 2020)

"the protection of any representation of information that permits the identity of an individual to whom the information applies to be reasonably inferred by either direct or indirect means." (James Kelly et al, "Data in the Wild: A KM Approach to Collecting Census Data Without Surveying the Population and the Issue of Data Privacy", 2020)

"A person’s right to control how much information about her/him/them is collected, used, shared by others." (Zerin M Khan, "How Do Mobile Applications for Cancer Communicate About Their Privacy Practices?: An Analysis of Privacy Policies", 2021)

"Deals defining what data may be lawfully shared with third parties, by an individual or organization." (Nikhil Padayachee & Surika Civilcharran, "Predicting Student Intention to Use Cloud Services for Educational Purposes Based on Perceived Security and Privacy", 2021)

"Is the aspect of information and communication technology that deals with the ability an organization or individual to determine what data and information in computer system can be shared with third parties." (Valerianus Hashiyana et al, "Integrated Big Data E-Healthcare Solutions to a Fragmented Health Information System in Namibia", 2021)