💎SQL Reloaded: Query Patterns in SQL Server (Inserts)

Independently on whether one is a database developer, administrator, or any other professional in the data area, as part of the job is important to be able to manipulate data on the fly. From the availability of DTS (Data Transformation Services) replaced since SQL Server 2005 with SSIS (SQL Server Integration Services), the use of packages for data import become one of the simplest and fastest ways of importing data into a database. It still is, however when dealing with small volume of data, it’s easier to import the data over a few SQL scripts, like in the below examples.
The old-fashion way of loading data is by defining first the target table based on the source’s definition:

-- Course data 
--DROP TABLE IF EXISTS dbo.T_Courses 

CREATE TABLE [dbo].[T_Courses](
     [CourseId] [int] NOT NULL,
     [CourseName] [varchar](50) NOT NULL
 ) ON [PRIMARY]

-- old-fashioned invidual inserts(SQL Server 2000)
INSERT INTO dbo.T_Courses
VALUES (1, 'Course 1')

INSERT INTO dbo.T_Courses(CourseId, CourseName)
VALUES (2, 'Course 2')

-- inserting multiple rows to a table at once (SQL Server 2005)
INSERT INTO dbo.T_Courses
--OUTPUT inserted.CourseId, inserted.CourseName -- 
VALUES (3, 'Course 3')
 , (4, 'Course 4')
 , (5, 'Course 5')
 , (6, 'Course 6')
 , (7, 'Course 7')

 SELECT *
 FROM dbo.T_Courses

Sometimes the definition of the table is not important. With SQL Server 2008 it was possible to create the table on the fly, and thus the inserts can be simplified as follows:

-- Course data 
--DROP TABLE IF EXISTS dbo.T_Courses 

SELECT *
INTO dbo.T_Courses
FROM (
  VALUES (1, 'Course 1')
 , (2, 'Course 2')
 , (3, 'Course 3')
 , (4, 'Course 4')
 , (5, 'Course 5')
 , (6, 'Course 6')
 , (7, 'Course 7')
 ) C (CourseId, CourseName)


 SELECT *
 FROM dbo.T_Courses

Now, to demonstrate further techniques it is useful to define besides Courses also a Students, respectively an Allocation table, the later storing the relation between a Course and a Student. As can be seen there is in the Students table a reference to the Courses, showing the course in which the student is enrolled currently  The model is only for demonstration purposes:

-- Student data with current courses
-- DROP TABLE IF EXISTS dbo.T_Students

SELECT StudentId
 , StudentName
 , CourseId
 , Cast(StartDate as Date) StartDate
 , Cast(EndDate as Date) EndDate
 INTO dbo.T_Students
 FROM (
  VALUES (1, 'Student 1', 1, '20170201', '20170228')
 , (2, 'Student 2', 1, '20170201', '20170228')
 , (3, 'Student 3', 2, '20170201', '20170228')
 , (4, 'Student 4', 3, '20170201', '20170228')
 , (5, 'Student 5', NULL, NULL, NULL)
 , (6, 'Student 6', NULL, NULL, NULL)
 , (7, 'Student 7', NULL, NULL, NULL)
 ) S (StudentId, StudentName, CourseId, StartDate, EndDate)


 SELECT *
 FROM dbo.T_Students

-- Course allocations 
-- DROP TABLE IF EXISTS dbo.T_Allocations

SELECT CourseId
 , StudentId
 , Cast(StartDate as Date) StartDate
 , Cast(EndDate as Date) EndDate
 , Hours
 , Mark 
 INTO dbo.T_Allocations
 FROM (
  VALUES  (1, 1, '20170101', '20170131', 4, 1)
 , (1, 2, '20170101', '20170131', 5, 2)
 , (1, 3, '20170101', '20170131', 12, 3)
 , (2, 1, '20170201', '20170128', 23, 1)
 , (2, 3, '20170201', '20170128', 12, 3)
 , (2, 4, '20170201', '20170128', 21, 1)
 , (1, 5, '20170201', '20170128', 12, 1)
 , (4, 1, '20170101', '20170131', 5, 2)
 , (4, 1, '20170101', '20170131', 4, 2)
 ) A(CourseId, StudentId, StartDate, EndDate, Hours, Mark)

ALTER TABLE dbo.T_Allocations 
ADD AllocationId int IDENTITY(1,1) 

SELECT *
FROM dbo.T_Allocations

With this the data model is ready for use. By the initial design the courses in which the Students are currents enrolled were not supposed to be into the Allocations table. The following query repairs this:

-- insert data from query
INSERT INTO dbo.T_Allocations (CourseId, StudentId, StartDate, EndDate, Hours)
SELECT C.CourseId
, C.StudentId
, C.StartDate
 , C.EndDate
 , 0 Hours
 FROM dbo.T_Students C
      LEFT JOIN dbo.T_Allocations A
        ON C.CourseId = A.CourseId
       AND C.StudentId = A.Studentid 
 WHERE A.CourseId IS NULL
   AND C.CourseId IS NOT NULL

If is needed to append more data, one can modify the corresponding query from above as follows:

-- insert data from values with check against existing data 
INSERT INTO dbo.T_Allocations
SELECT DAT.CourseId
 , DAT.StudentId
 , Cast(DAT.StartDate as Date) StartDate
 , Cast(DAT.EndDate as Date) EndDate
 , DAT.Hours
 , DAT.Mark 
 FROM (
  VALUES  (5, 5, '20170101', '20170131', 4, 2)
 , (5, 5, '20170101', '20170131', 5, 3)
 , (5, 5, '20170101', '20170131', 12, 4)
 , (4, 1, '20170101', '20170131', 4, 2) --already exists 
 ) DAT(CourseId, StudentId, StartDate, EndDate, Hours, Mark)
   LEFT JOIN dbo.T_Allocations A
     ON DAT.CourseId = A.CourseId
    AND DAT.StudentId = A.Studentid 
 WHERE A.CourseId IS NULL

Starting with the 2008 version, a MERGE statement was introduced in SQL Server, which allows adding further behavior within the same statement for INSERTS, UPDATES or DELETES. The above insert based on a MERGE becomes:

-- insert data via a merge
 MERGE INTO dbo.T_Allocations A USING
 (
     SELECT DAT.CourseId
     , DAT.StudentId
     , Cast(DAT.StartDate as Date) StartDate
     , Cast(DAT.EndDate as Date) EndDate
     , DAT.Hours
     , DAT.Mark 
     FROM (
      VALUES  (5, 5, '20170101', '20170131', 4, 2)
     , (5, 5, '20170101', '20170131', 5, 3)
     , (5, 5, '20170101', '20170131', 12, 4)
     , (4, 1, '20170101', '20170131', 4, 2) --already exists 
     ) DAT(CourseId, StudentId, StartDate, EndDate, Hours, Mark)
 ) DAT
   ON DAT.CourseId = A.CourseId
  AND DAT.StudentId = A.Studentid 
 WHEN NOT MATCHED THEN 
 INSERT (CourseId, StudentId, StartDate, EndDate, Hours, Mark)
 VALUES (DAT.CourseId, DAT.StudentId, DAT.StartDate, DAT.EndDate, DAT.Hours, DAT.Mark);

These are the types of scripts I typically use to insert data into a database, and this independently of tables’ size. For more check Microsoft’s documentation on INSERT. The preparation of the data to be imported can be easily done in Excel with a few macros.

#️⃣☯Software Engineering: Concept Documents (The Good, the Bad and the Ugly)

A concept document (simply a concept) is a document that describes at high level the set of necessary steps and their implications in order to achieve a desired result, typically making the object of a project. In other words, it describes how something can be done or achieved, respectively how a problem can be solved.

The Good: The main aim of the document is to give all the important aspects and to assure that the idea is worthy of consideration, that the steps considered provide a good basis for further work, respectively to provide a good understanding for the various parties involved, Therefore, concepts are used as a basis for the sign-off, respectively for the implementation of software and hardware solutions.

 A concept provides information about the context, design, architecture, security, usage, purpose and/or objectives of the future solution together with the set of assumptions, constraints and implications. A concept is not necessarily a recipe because it attempts providing a solution for a given problem or situation that needs a solution. Even if it bears many similarities in content and structure a concept it also not a strategy, because the strategy offers an interpretation of the problem, and also not a business case, because the later focuses mainly on the financial aspects.

A concept proves thus to be a good basis for implementing the described solution, being often an important enabler. On the other side, a written concept is not always necessary, even if conceptualization must exist in implementers’ head.

The Bad: From these considerations projects often consider the elaboration of a concept before further work can be attempted. To write such a document is needed to understand the problem/situation and be capable of sketching a solution in which the various steps or components fit together as the pieces of a puzzle. The problem is that the more complex the problem to be solved, the fuzzier the view and understanding of the various pieces becomes, respectively, the more challenging it becomes to fit the pieces together. In certain situations, it becomes almost impossible for a single person to understand and handle all the pieces. Solving the puzzle becomes a collective approach where the complexity is broken in manageable parts in the detriment of other aspects.

Writing a concept is a time-consuming task. The more accuracy and details are needed, the longer it takes to write and review the document, time that’s usually stolen from other project phases, especially when the phases are considered as sequential. It takes about 20% from the total effort needed to write a ‘perfect’ concept for writing a concept that covers only 80% of the facts, while 80% from the effort to consider the remaining 20% of the facts as the later involve multiple iterations. In extremis, aiming for perfection will make one start the implementation late or not start at all. It’s a not understandable pedantry with an important impact on projects'
 timeline and quality in the hope of a quality increase, which is sometimes even illusory.

The Ugly: The concept-based approach is brought to extreme in ERP implementations where for each process or business area is needed to write a concept, which often carries fancy names – solution design document, technical design document, business process document, etc. Independently how it is called, the purpose is to describe how the solution is implemented. The problem is that the conceptualization phase tends to take much longer than planned given the dependencies between the various business area in terms of functionality and activities. The complexity can become overwhelming, with an important impact on project’s budget, time and quality.

💫🧮☯ERP: Microsoft Dynamics 365 (The Good, the Bad and the Ugly)

The Good: The shift made by Microsoft by porting their Dynamics AX ERP solution to a web-based application (aka D365) hosted in the Microsoft cloud, offered them a boost on the ERP market. The integration with the Office and BI stack, as well Microsoft’s One Version strategy of pushing new features almost on a monthly basis, and of having customers at a maximum 2 releases from the current version, makes from D365 a solution to consider for small to big organizations that span over business sectors and geographies.

The Bad: Currently the monthly release cycle seems to be a challenge for the customers and service providers altogether. Even if the changes in existing functionality are minor, while the functionality is thoroughly tested before releases, the customers still need to test the releases in several systems, especially to assure that the customizations and integrations still work. This can prove to be quite a challenge in which automatic or semiautomatic tools can help when adequately used. Even then, a considerable effort needs to be addressed by the parties involved.

The burden is bigger for the service providers that build their own solutions for D365 as they need to assure in advance that after each release the applications still work. From customers’ perspective, the more such applications they use, the higher the risks of delays in adopting a release or, in extremis, to look for similar solutions. In theory, with good planning and by following best practices the risks are small, though that’s just the theory speaking.

If in the past 2-3 instances were enough to support the ERP during and post implementation, currently the requirements for the cloud-based solution more than doubled, an organization arriving to rent 5-7 D365 instances for the same. Moreover, even if the split between the main blocks (Finance, Supply Chain, Retail and Talent), plus the various Customer Engagement packages, provides some flexibility when thy are combined, this leads to a considerable price increase. Further costs are related to the gaps existing in the available functionality. More likely Microsoft will attempt closing some of the gaps, however until then the customers are forced to opt for existing solutions or have the functionality built. Microsoft pretends that their cloud-based ERP solution provides lower ownership costs, however, looking at the prices, it’s questionable on whether D365 is affordable for small and average organizations. To put it bluntly – think how many socks (aka products) one needs to sell just to cover the implementation, the licensing and infrastructure costs!

One important decision taken by Microsoft was to not allow the direct access to the D365 production database, decision that limits an organization’s choices and flexibility in addressing reporting requirements. Of course, the existing BI infrastructure can still be leveraged with a few workarounds, though the flexibility is lost, while further challenges are involved.

The Ugly: ERP implementations based on D365 make no exceptions from the general trend – given their complexity they are predisposed to fail achieving the set objectives, and this despite Microsoft’s attempts of creating methodologies, tools and strong communities to support the service providers and customers in such projects. The reasons for failure reside with the customers and service providers altogether, the chains of implications forming a complex network of causalities with multiple levels of reinforcement. When the negative reinforcements break the balance, it can result a tipping point where the things start to go wrong – escalations, finger-pointing, teams’ restructuring, litigations, etc. In extremis, even if the project reaches the finish, the costs can easily reach an overrun of 50-150% from the initial estimation, and that’s a lot to bear.

💼☯Project Management: Methodologies (The Good, the Bad and the Ugly)

The Good: Nowadays there're several Project Management (PM) methodologies to choose from to address a project’s specifics and, when adapted and applied accordingly, a methodology can enable projects to be run and brought under control.

The Bad: Even if the theoretical basis of PM methodologies has been proved and perfected over the years, projects continue to fail at a disturbing rate. Of course, the reasons behind their failure are multiple, though often the failure reasons are rooted in how PM methodologies are taught, understood and implemented.

Same as a theoretical course in cooking won’t make one a good cook, a theoretical course in PM won’t make one a good Project Manager or knowledgeable team member in applying the learned methodology. Surprisingly, the expectation is exactly that – the team member got a training and is good to go. Moreover, people believe that managing a software project is like coordinating the building of a small treehouse. To some degree there are many similarities though the challenges typically lie in details, and these details often escape a standard course.

To bridge the gap between theory and practice is needed time for the learner to grow in the role, to learn the does and don’ts, and, most important, to learn how to use the tools at hand efficiently. The methodology is itself a tool making use of further tools in its processes – project plans, work breakdown structures, checklists, charters, reports, records, etc. These can be learned only through practice, hopefully with some help (aka mentoring) from an experienced person in the respective methodology, either the Project Manager itself, a trainer or other team member. Same as one can’t be thrown into the water and expected to traverse the Channel Tunnel, you can’t do that with a newbie.

There’s a natural fallacy to think that we've understood more than we have. We can observe our understanding's limits when we are confronted with the complexities involved in handing PM activities. A second fallacy is not believing other people’s warnings against using a tool or performing an activity in a certain way. A newbie’s mind has sometimes the predisposition of a child to try touching a hot stove even if warned against it. It’s part of the learning process, though some persist in such behavior without learning much. What’s even more dangerous is a newbie pretending to be an expert and this almost always ends badly.

The Ugly appears when the bad is brought to extreme, when methodologies are misused for the wrong purposes to the degree that they destroy anything in their way. Of course, a pool can be dug by using a spoon but does it make sense to do that? Just because a tool can be used for something it doesn’t mean it should be used for it as long there are better tools for the same. It seems a pretty logical thing though the contrary happens more often than we’d like. It starts with the preconception that one should use the tool one knows best, ignoring in the process the fit for purpose condition. What’s even more deplorable is breaking down a project to fit a methodology while ignoring the technical and logistical aspects.

Any tool can lead to damages when used excessively, in wrong places, at the wrong point in time or by the wrong person. Like the instruments in an orchestra, when an instrument plays the wrong note, it dissonates from the rest. When more instruments play wrongly, then the piece is unrecognizable. It’s the role of the bandmaster to make the players touch the right notes at the right time.

🗄️Data Management: Data Literacy (Part I: A Second Language)

At the Gartner Data & Analytics Summit that took place in 2018 in Grapevine, Texas, it was reiterated the importance of data literacy for taking advantage of the emergence of data analytics, artificial intelligence (AI) and machine learning (ML) technologies. Gartner expected then that by 2020, 80% of organizations will initiate deliberate competency development in the field of data literacy [1] – or how they put it – learning to ‘speak data’ as a ‘second language’.

Data literacy is typically defined as the ability to read, work with, analyze, and argue with data. Sure, these form the blocks of data literacy, though what I’m missing from this definition is the ability to understand the data, even if understanding should be the outcome of reading, and the ability to put data into the context of business problems, even if the analyzes of data could involve this later aspect too.

Understanding has several aspects: understanding the data structures available within an organization, understanding the problems with data (including quality, governance, privacy and security), respectively understanding how the data are linked to the business processes. These aspects go beyond the simple ability included in the above definition, which from my perspective doesn’t include the particularities of an organization (data structure, data quality and processes) – the business component. This is reflected in one of the problems often met in the BI/data analytics industry – the solutions developed by the various service providers don’t reflect organizations’ needs, one of the causes being the inability to understand the business on segments or holistically.  

Putting data into context means being able to use the respective data in answering stringent business problems. A business problem needs to be first correctly defined and this requires a deep understanding of the business. Then one needs to identify the data that could help finding the answers to the problem, respectively of building one or more models that would allow elaborating further theories and performing further simulations. This is an ongoing process in which the models built are further enhanced, when possible, or replaced by better ones.

Probably the comparison with a second language is only partially true. One can learn a second language and argue in the respective language, though it doesn’t mean that the argumentations will be correct or constructive as long the person can’t do the same in the native language. Moreover, one can have such abilities in the native or a secondary language, but not be able do the same in what concerns the data, as different skillsets are involved. This aspect can make quite a difference in a business scenario. One must be able also to philosophize, think critically, as well to understand the forms of communication and their rules in respect to data.

To philosophize means being able to understand the causality and further relations existing within the business and think critically about them. Being able to communicate means more than being able to argue – it means being able to use effectively the communication tools – communication channels, as well the methods of representing data, information and knowledge. In extremis one might even go beyond the basic statistical tools, stepping thus in what statistical literacy is about. In fact, the difference between the two types of literacy became thinner, the difference residing in the accent put on their specific aspects.

These are the areas which probably many professionals lack. Data literacy should be the aim, however this takes time and is a continuous iterative process that can take years to reach maturity. It’s important for organizations to start addressing these aspects, progress in small increments and learn from the experience accumulated.

Previous Post <<||>> Next Post

References:
[1] Gartner (2018) How data and analytics leaders learn to master information as a second language, by Christy Pettey (link) 

#️⃣Software Engineering: Mea Culpa (Part II: The Beginnings)

Software Engineering Series

I started programming at 14-15 years old with logical schemas made on paper, based mainly on simple mathematical algorithms like solving equations of second degree, finding prime or special numbers, and other simple tricks from the mathematical world available for a student at that age. It was challenging to learn programming based only on schemas, though, looking back, I think it was the best learning basis a programmer could have, because it allowed me thinking logically and it was also a good exercise, as one was forced to validate mentally or on paper the outputs.

Then I moved to learning Basic and later Pascal on old generation Spectrum computers, mainly having a keyboard with 64K memory and an improvised monitor. It felt almost like a holiday when one had the chance to work 45 minutes or so on an IBM computer with just 640K memory. It was also a motivation to stay long after hours to write a few more lines of code. Even if it made no big difference in what concerns the speed, the simple idea of using a more advanced computer was a big deal.

The jump from logical schemas to actual programming was huge, as we moved from static formulas to exploratory methods like the ones of finding the roots of equations of upper degrees by using approximation methods, working with permutations and a few other combinatoric tools, interpolation methods, and so on. Once I got my own 64K Spectrum keyboard, a new world opened, having more time to play with 2- and 3-dimensional figures, location problems and so on. It was probably the time I got most interesting exposure to things not found in the curricula.  

Further on, during the university years I moved to Fortran, back to Pascal and dBASE, and later to C and C++, the focus being further on mathematical and sorting algorithms, working with matrices, and so on. I have to admit that it was a big difference between the students who came from 2-3 hours of Informatics per week (like I did) and the ones coming from lyceums specialized on Informatics, this especially during years in which learning materials were almost inexistent. In the end all went well.

The jumping through so many programming languages, some quite old for the respective times, even if allowed acquiring different perspectives, it felt sometimes like  a waste of time, especially when one was limited to using the campus computers, and that only during lab hours. That was the reality of those times. Fortunately, the university years went faster than they came. Almost one year after graduation, with a little help, some effort and benevolence, I managed to land a job as web developer, jumping from an interlude with Java to ASP, JavaScript, HTML, ColdFusion, ActionScript, SQL, XML and a few other programming languages ‘en vogue’ during the 2000.

Somewhere between graduation and my first job, my life changed when I was able to buy my own PC (a Pentium). It was the best investment I could make, mainly because it allowed me to be independent of what I was doing at work. It allowed me learning the basics of OOP programming based on Visual Basic and occasionally on Visual C++ and C#. Most of the meaningful learning happened after work, from the few books available, full of mistakes and other challenges.

That was my beginning. It is not my intent to brag about how much or how many programming languages I learned - knowledge is anyway relative - but to differentiate between the realities of then and today, as a bridge over time.

Previous Post <<||>>  Next Post

💠SQL Server: Using the R Language in SQL Server 2016 (Hello World & Working with Data Frames) [new feature]

One of the most interesting features coming with SQL Server 2016 is the possibility to run external scripts written in the R language or Python, taking thus advantage of the numerical and statistical packages coming with the respective languages. The next examples are based on the R language.

As the scripts in R are considered as external scripts, is needed first to enable the 'external scripts enabled' configuration option by using the following script (a server restart is required):

-- enable external scripts 
sp_configure 'external scripts enabled', 1;
RECONFIGURE WITH OVERRIDE;

To makes sure that the functionality works as expected, it makes sense to attempt first a "hello world" example:

-- hello world script
EXECUTE sp_execute_external_script 

        @language = N'R',  
        @script = N'print("Hello world")'

The R language come with a few predefined datasets and for the following examples I’ll be using the mtcars dataset. Because the dataset contains several columns, I will use only the first 3. For this one care run a script as the following into the R console:

Usually it’s useful to look first at the structure of the dataset, this by using str(mtcars) command in the R console:

To return the dataset from R one can use the following call to the sp_execute_external_script stored procedure:

  -- returning the first 3 columns 
 EXEC sp_execute_external_script  
       @language = N'R'  
     , @script = N'cars <- mtcars[1:3];'
     , @input_data_1 = N''  
     , @output_data_1_name = N'cars'
     WITH RESULT SETS (("mpg" float not null 
         , "cyl" float not null 
         , "disp" float not null 
       ));  

As can be seen, besides the script is needed to define a variable in which the returning dataset is stored, as well the resulting dataset. Unfortunately, this script doesn’t return rows’ names. To do that I had to use a small trick by concatenating the initial data frame with the one resulting from row’s names. (Please let me know if you have another way of achieving the same.)

The script becomes:

 -- returning the first 3 columns including rows' name
 EXEC   sp_execute_external_script  
       @language = N'R'  
     , @script = N'cars <- data.frame(rownames(mtcars), mtcars[1:3]);'
     , @input_data_1 = N''  
     , @output_data_1_name = N'cars'
     WITH RESULT SETS (("Car" varchar(100)
      , "mpg" float not null 
      , "cyl" float not null 
      , "disp" float not null 
       ));  

To reuse the script, it can be included in a stored procedure, similarly like the examples provided by the Microsoft documentation for the sp_execute_external_script stored procedure.

A few pointers:
1. It’s useful to test your scripts first in the R console.
2. If 'external scripts enabled' was not enabled, then the following error message will appear:
Msg 39023, Level 16, State 1, Procedure sp_execute_external_script, Line 1 [Batch Start Line 22]
'sp_execute_external_script' is disabled on this instance of SQL Server. Use sp_configure 'external scripts enabled' to enable it.
3. It might be needed to start the “SQL Server Launchpad” service manually, a hint in this direction comes from the following error message:
Msg 39011, Level 16, State 1, Line 24
SQL Server was unable to communicate with the LaunchPad service. Please verify the configuration of the service. 
4. Once the examples tested, it might be recommended to disable the 'external scripts enabled' configuration option as long is not needed anymore.
5. Check the various ways to analyse the mtcars dataset using the R language: https://rpubs.com/BillB/217355

📊R Language: Drawing Function Plots (Part I - Basic Curves)

Besides the fact that is free, one of the advantages of the R language is that it allows drawing function plots with simple commands. All one needs is software package like Microsoft R Open and one is set to go.

For example, to draw the function f(x) = x^3-3*x+2 all one needs to do is to type the following command into the console:

curve(x^3-3*x+2, -3,3)

One can display a second function into the same chart by adding a second command and repeat the process further as needed:

curve(x^2-3*x+2, add=TRUE, col="blue")
curve(x^4-3*x+2, add=TRUE, col="red")
curve(x^5-3*x+2, add=TRUE, col="green")

As one can see a pattern emerges already …

One could easily display the plots in their one section by defining an array on the display device, allowing thus to focus on special characteristics of the functions:

par(mfrow=c(2,2))
curve(x^3-3*x+2, -3,3)
curve(x^2-3*x+2, -3,3, col="blue")
curve(x^4-3*x+2, -3,3, col="red")
curve(x^5-3*x+2, -3,3, col="green")

One can be creative and display the functions using a loop:

par(mfrow=c(2,2))
for(n in c(2:5)){
curve(x^n-3*x+2, -3,3)
}

Similarly, one can plot trigonometric functions:

par(mfrow=c(2,3))
curve(sin(x),-pi,pi) 
curve(cos(x),-pi,pi) 
curve(tan(x),-pi,pi) 
curve(1/tan(x),-pi,pi) 
curve(asin(x),-1,1)
curve(acos(x),-1,1)

The possibilities are endless. For complex functions one can include the function into an r function:

myFunction<- function(x) sin(cos(x)*exp(-x/2))
curve(myFunction, -15, 15, n=1000)

For the SQL Server developers, what’s even greater is the possibility of using Management Studio for the same, though that’s a topic for another post.

 

Happy coding!

Previous Post <<||>> Next Post

💻IT: Technology (Just the Quotes)

"Systems engineering embraces every scientific and technical concept known, including economics, management, operations, maintenance, etc. It is the job of integrating an entire problem or problem to arrive at one overall answer, and the breaking down of this answer into defined units which are selected to function compatibly to achieve the specified objectives. [...] Instrument and control engineering is but one aspect of systems engineering - a vitally important and highly publicized aspect, because the ability to create automatic controls within overall systems has made it possible to achieve objectives never before attainable, While automatic controls are vital to systems which are to be controlled, every aspect of a system is essential. Systems engineering is unbiased, it demands only what is logically required. Control engineers have been the leaders in pulling together a systems approach in the various technologies." (Instrumentation Technology, 1957)

"Doing engineering is practicing the art of the organized forcing of technological change." (George Spencer-Brown, Electronics, Vol. 32 (47),  1959)

"The decision which achieves organization objectives must be both (1) technologically sound and (2) carried out by people. If we lose sight of the second requirement or if we assume naively that people can be made to carry out whatever decisions are technically soundwe run the risk of decreasing rather than increasing the effectiveness of the organization." (Douglas McGregor, "The Human Side of Enterprise", 1960)

"Any sufficiently advanced technology is indistinguishable from magic." (Arthur C Clarke, "Profiles of the Future: An Inquiry into the Limits of the Possible", 1962)

"Science is the reduction of the bewildering diversity of unique events to manageable uniformity within one of a number of symbol systems, and technology is the art of using these symbol systems so as to control and organize unique events. Scientific observation is always a viewing of things through the refracting medium of a symbol system, and technological praxis is always handling of things in ways that some symbol system has dictated. Education in science and technology is essentially education on the symbol level." (Aldous L Huxley, "Essay", Daedalus, 1962)

"Engineering is the art of skillful approximation; the practice of gamesmanship in the highest form. In the end it is a method broad enough to tame the unknown, a means of combing disciplined judgment with intuition, courage with responsibility, and scientific competence within the practical aspects of time, of cost, and of talent. This is the exciting view of modern-day engineering that a vigorous profession can insist be the theme for education and training of its youth. It is an outlook that generates its strength and its grandeur not in the discovery of facts but in their application; not in receiving, but in giving. It is an outlook that requires many tools of science and the ability to manipulate them intelligently In the end, it is a welding of theory and practice to build an early, strong, and useful result. Except as a valuable discipline of the mind, a formal education in technology is sterile until it is applied." (Ronald B Smith, "Professional Responsibility of Engineering", Mechanical Engineering Vol. 86 (1), 1964)

"It is a commonplace of modern technology that there is a high measure of certainty that problems have solutions before there is knowledge of how they are to be solved." (John K Galbraith, "The New Industrial State", 1967)

"In many ways, project management is similar to functional or traditional management. The project manager, however, may have to accomplish his ends through the efforts of individuals who are paid and promoted by someone else in the chain of command. The pacing factor in acquiring a new plant, in building a bridge, or in developing a new product is often not technology, but management. The technology to accomplish an ad hoc project may be in hand but cannot be put to proper use because the approach to the management is inadequate and unrealistic. Too often this failure can be attributed to an attempt to fit the project to an existing management organization, rather than molding the management to fit the needs of the project. The project manager, therefore, is somewhat of a maverick in the business world. No set pattern exists by which he can operate. His philosophy of management may depart radically from traditional theory." (David I Cleland & William R King, "Systems Analysis and Project Management", 1968)

"Technological invention and innovation are the business of engineering. They are embodied in engineering change." (Daniel V DeSimone & Hardy Cross, "Education for Innovation", 1968)

"Advanced technology required the collaboration of diverse professions and organizations, often with ambiguous or highly interdependent jurisdictions. In such situations, many of our highly touted rational management techniques break down; and new non-engineering approaches are necessary for the solution of these 'systems' problems." (Leonard R Sayles &Margaret K Chandler, "Managing Large Systems: The Large-Scale Approach", 1971)

"It follows from this that man's most urgent and pre-emptive need is maximally to utilize cybernetic science and computer technology within a general systems framework, to build a meta-systemic reality which is now only dimly envisaged. Intelligent and purposeful application of rapidly developing telecommunications and teleprocessing technology should make possible a degree of worldwide value consensus heretofore unrealizable." (Richard F Ericson, "Visions of Cybernetic Organizations", 1972)

"Technology can relieve the symptoms of a problem without affecting the underlying causes. Faith in technology as the ultimate solution to all problems can thus divert our attention from the most fundamental problem - the problem of growth in a finite system." (Donella A Meadows, "The Limits to Growth", 1972)

"Modern scientific principle has been drawn from the investigation of natural laws, technology has developed from the experience of doing, and the two have been combined by means of mathematical system to form what we call engineering." (George S Emmerson, "Engineering Education: A Social History", 1973)

"The system of nature, of which man is a part, tends to be self-balancing, self-adjusting, self-cleansing. Not so with technology." (Ernst F Schumacher, "Small is Beautiful", 1973)

"Above all, innovation is not invention. It is a term of economics rather than of technology. [...] The measure of innovation is the impact on the environment. [...] To manage innovation, a manager has to be at least literate with respect to the dynamics of innovation." (Peter F Drucker, "People and Performance", 1977)

"Numeracy has two facets-reading and writing, or extracting numerical information and presenting it. The skills of data presentation may at first seem ad hoc and judgmental, a matter of style rather than of technology, but certain aspects can be formalized into explicit rules, the equivalent of elementary syntax." (Andrew Ehrenberg, "Rudiments of Numeracy", Journal of Royal Statistical Society, 1977)

"Engineering or Technology is the making of things that did not previously exist, whereas science is the discovering of things that have long existed." (David Billington, "The Tower and the Bridge: The New Art of Structural Engineering", 1983)

"No matter how high or how excellent technology may be and how much capital may be accumulated, unless the group of human beings which comprise the enterprise works together toward one unified goal, the enterprise is sure to go down the path of decline." (Takashi Ishihara, Cherry Blossoms and Robotics, 1983)

"People’s views of the world, of themselves, of their own capabilities, and of the tasks that they are asked to perform, or topics they are asked to learn, depend heavily on the conceptualizations that they bring to the task. In interacting with the environment, with others, and with the artifacts of technology, people form internal, mental models of themselves and of the things with which they are interacting. These models provide predictive and explanatory power for understanding the interaction." (Donald A Norman, "Some observations on Mental Models", 1983)

"With the changes in technological complexity, especially in information technology, the leadership task has changed. Leadership in a networked organization is a fundamentally different thing from leadership in a traditional hierarchy." (Edgar Schein, "Organizational Culture and Leadership", 1985)

"[Computer and other technical managers] must become business managers or risk landing on the technological rubbish heap." (Jim Leeke, PC Week, 1987)

"Most managers are not capable of making decisions involving complex technological matters without help - lots of it. [...] The finest technical people on the job should have a dual role: doing technical work and advising management." (Philip W Metzger, "Managing Programming People", 1987)

"People don't want to understand all the components; they just want to make it [the technology] happen." (Bernadine Nicodemus, PC Week, 1987)

"The major problems of our work are not so much technological as sociological in nature. Most managers are willing to concede the idea that they’​​​​​​ve got more people worries than technical worries. But they seldom manage that way. They manage as though technology were their principal concern. They spend their time puzzling over the most convoluted and most interesting puzzles that their people will have to solve, almost as though they themselves were going to do the work rather than manage it. […] The main reason we tend to focus on the technical rather than the human side of the work is not because it’​​​​​​s more crucial, but because it’​​​​​​s easier to do." (Tom DeMarco & Timothy Lister, "Peopleware: Productive Projects and Teams", 1987)

"Information technology can capture and process data, and expert systems can to some extent supply knowledge, enabling people to make their own decisions. As the doers become self-managing and self-controlling, hierarchy - and the slowness and bureaucracy associated with it - disappears." (Michael M Hammer, "Reengineering Work: Don't Automate, Obliterate", Magazine, 1990) [source]

"The new information technologies can be seen to drive societies toward increasingly dynamic high-energy regions further and further from thermodynamical equilibrium, characterized by decreasing specific entropy and increasingly dense free-energy flows, accessed and processed by more and more complex social, economic, and political structures." (Ervin László, "Information Technology and Social Change: An Evolutionary Systems Analysis", Behavioral Science 37, 1992)

"Ignorance of science and technology is becoming the ultimate self-indulgent luxury." (Jeremy Bernstein, "Cranks, Quarks, and the Cosmos: Writings on Science", 1993)

"Technology is nothing. What’s important is that you have a faith in people, that they’re basically good and smart, and if you give them tools, they’ll do wonderful things with them." (Steve Jobs, Rolling Stone, 1994)

"Now that knowledge is taking the place of capital as the driving force in organizations worldwide, it is all too easy to confuse data with knowledge and information technology with information." (Peter Drucker, "Managing in a Time of Great Change", 1995)

"Commonly, the threats to strategy are seen to emanate from outside a company because of changes in technology or the behavior of competitors. Although external changes can be the problem, the greater threat to strategy often comes from within. A sound strategy is undermined by a misguided view of competition, by organizational failures, and, especially, by the desire to grow." (Michael E Porter, "What is Strategy?", Harvard Business Review, 1996)

"Management is a set of processes that can keep a complicated system of people and technology running smoothly. The most important aspects of management include planning, budgeting, organizing, staffing, controlling, and problem solving. Leadership is a set of processes that creates organizations in the first place or adapts them to significantly changing circumstances. Leadership defines what the future should look like, aligns people with that vision, and inspires them to make it happen despite the obstacles." (John P Kotter, "Leading Change", 1996)

"Networks constitute the new social morphology of our societies, and the diffusion of networking logic substantially modifies the operation and outcomes in processes of production, experience, power, and culture. While the networking form of social organization has existed in other times and spaces, the new information technology paradigm provides the material basis for its pervasive expansion throughout the entire social structure." (Manuel Castells, "The Rise of the Network Society", 1996)

"Issues of quality, timeliness and change are the conditions that are forcing us to face up to the issues of enterprise architecture. The precedent of all the older disciplines known today establishes the concept of architecture as central to the ability to produce quality and timely results and to manage change in complex products. Architecture is the cornerstone for containing enterprise frustration and leveraging technology innovations to fulfill the expectations of a viable and dynamic Information Age enterprise." (John Zachman, "Enterprise Architecture: The Issue of The Century", 1997)

"The Enterprise Architecture is the explicit description of the current and desired relationships among business and management process and information technology. It describes the 'target' situation which the agency wishes to create and maintain by managing its IT portfolio." (Franklin D Raines, 1997)

"All things being equal, choose technology that connects. […] This aspect of technology has increasing importance, at times overshadowing such standbys as speed and price. If you are in doubt about what technology to purchase, get the stuff that will connect the most widely, the most often, and in the most ways. Avoid anything that resembles an island, no matter how well endowed that island is." (Kevin Kelly, "New Rules for the New Economy: 10 radical strategies for a connected world", 1998)

"Beauty is more important in computing than anywhere else in technology because software is so complicated. Beauty is the ultimate defense against complexity." (David Gelernter, "Machine Beauty: Elegance And The Heart Of Technolog", 1998)

"Modelling techniques on powerful computers allow us to simulate the behaviour of complex systems without having to understand them.  We can do with technology what we cannot do with science.  […] The rise of powerful technology is not an unconditional blessing.  We have  to deal with what we do not understand, and that demands new  ways of thinking." (Paul Cilliers,"Complexity and Postmodernism: Understanding Complex Systems", 1998)

"Technology is no panacea. It will never solve the ills or injustices of society. Technology can do only one thing for us - but it is an astonishing thing: Technology brings us an increase in opportunities." (Kevin Kelly, "New Rules for the New Economy: 10 radical strategies for a connected world", 1998)

"A primary reason that evolution - of life-forms or technology - speeds up is that it builds on its own increasing order." (Ray Kurzweil, "The Age of Spiritual Machines: When Computers Exceed Human Intelligence", 1999) 

"As systems became more varied and more complex, we find that no single methodology suffices to deal with them. This is particularly true of what may be called information intelligent systems - systems which form the core of modern technology. To conceive, design, analyze and use such systems we frequently have to employ the totality of tools that are available. Among such tools are the techniques centered on fuzzy logic, neurocomputing, evolutionary computing, probabilistic computing and related methodologies. It is this conclusion that formed the genesis of the concept of soft computing." (Lotfi A Zadeh, "The Birth and Evolution of Fuzzy Logic: A personal perspective", 1999)

"Enterprise architecture is a family of related architecture components. This include information architecture, organization and business process architecture, and information technology architecture. Each consists of architectural representations, definitions of architecture entities, their relationships, and specification of function and purpose. Enterprise architecture guides the construction and development of business organizations and business processes, and the construction and development of supporting information systems." (Gordon B Davis, "The Blackwell encyclopedic dictionary of management information systems"‎, 1999)

"Enterprise architecture is a holistic representation of all the components of the enterprise and the use of graphics and schemes are used to emphasize all parts of the enterprise, and how they are interrelated. [...] Enterprise architectures are used to deal with intra-organizational processes, interorganizational cooperation and coordination, and their shared use of information and information technologies. Business developments, such as outsourcing, partnership, alliances and Electronic Data Interchange, extend the need for architecture across company boundaries." (Gordon B Davis," The Blackwell encyclopedic dictionary of management information systems"‎, 1999)

"We do not learn much from looking at a model - we learn more from building the model and manipulating it. Just as one needs to use or observe the use of a hammer in order to really understand its function, similarly, models have to be used before they will give up their secrets. In this sense, they have the quality of a technology - the power of the model only becomes apparent in the context of its use." (Margaret Morrison & Mary S Morgan, "Models as mediating instruments", 1999)

"Periods of rapid change and high exponential growth do not, typically, last long. A new equilibrium with a new dominant technology and/or competitor is likely to be established before long. Periods of punctuation are therefore exciting and exhibit unusual uncertainty. The payoff from establishing a dominant position in this short time is therefore extraordinarily high. Dominance is more likely to come from skill in marketing and positioning than from superior technology itself." (Richar Koch, "The Power Laws", 2000)

"The business changes. The technology changes. The team changes. The team members change. The problem isn't change, per se, because change is going to happen; the problem, rather, is the inability to cope with change when it comes." (Kent Beck, "Extreme Programming Explained", 2000)

"A well-functioning team of adequate people will complete a project almost regardless of the process or technology they are asked to use (although the process and technology may help or hinder them along the way)." (Alistair Cockburn, "Agile Software Development", 2001)

"An Enterprise Architecture is a dynamic and powerful tool that helps organisations understand their own structure and the way they work. It provides a ‘map’ of the enterprise and a ‘route planner’ for business and technology change. A well-constructed Enterprise Architecture provides a foundation for the ‘Agile’ business." (Bob Jarvis, "Enterprise Architecture: Understanding the Bigger Picture - A Best Practice Guide for Decision Makers in IT", 2003)

"Normally an EA takes the form of a comprehensive set of cohesive models that describe the structure and functions of an enterprise. An important use is in systematic IT planning and architecting, and in enhanced decision-making. The EA can be regarded as the ‘master architecture’ that contains all the subarchitectures for an enterprise. The individual models in an EA are arranged in a logical manner that provides an ever-increasing level of detail about the enterprise: its objectives and goals; its processes and organisation; its systems and data; the technology used and any other relevant spheres of interest." (Bob Jarvis, "Enterprise Architecture: Understanding the Bigger Picture - A Best Practice Guide for Decision Makers in IT", 2003)

"Technology can relieve the symptoms of a problem without affecting the underlying causes. Faith in technology as the ultimate solution to all problems can thus divert our attention from the most fundamental problem - the problem of growth in a finite system - and prevent us from taking effective action to solve it." (Donella H Meadows & Dennis L Meadows, "The Limits to Growth: The 30 Year Update", 2004)

"To turn really interesting ideas and fledgling technologies into a company that can continue to innovate for years, it requires a lot of disciplines."  (Steve Jobs, BusinessWeek, 2004)

"You need a very product-oriented culture, even in a technology company. Lots of companies have tons of great engineers and smart people. But ultimately, there needs to be some gravitational force that pulls it all together. Otherwise, you can get great pieces of technology all floating around the universe." (Steve Jobs, Newsweek, 2004)

"Although the Singularity has many faces, its most important implication is this: our technology will match and then vastly exceed the refinement and suppleness of what we regard as the best of human traits." (Ray Kurzweil, "The Singularity is Near", 2005)

"The Singularity will represent the culmination of the merger of our biological thinking and existence with our technology, resulting in a world that is still human but that transcends our biological roots. There will be no distinction, post-Singularity, between human and machine or between physical and virtual reality. If you wonder what will remain unequivocally human in such a world, it’s simply this quality: ours is the species that inherently seeks to extend its physical and mental reach beyond current limitations." (Ray Kurzweil, "The Singularity is Near", 2005)

"Businesses are themselves a form of design. The design of a business encompasses its strategy, organizational structure, management processes, culture, and a host of other factors. Business designs evolve over time through a process of differentiation, selection, and amplification, with the market as the ultimate arbiter of fitness [...] the three-way coevolution of physical technologies, social technologies, and business designs [...] accounts for the patterns of change and growth we see in the economy." (Eric D Beinhocker, "The Origin of Wealth. Evolution, complexity, and the radical remaking of economics", 2006)

"Enterprise architecture is the organizing logic for business processes and IT infrastructure reflecting the integration and standardization requirements of a company's operation model. […] The key to effective enterprise architecture is to identify the processes, data, technology, and customer interfaces that take the operating model from vision to reality." (Jeanne W Ross et al, "Enterprise architecture as strategy: creating a foundation for business", 2006)

"Chance is just as real as causation; both are modes of becoming.  The way to model a random process is to enrich the mathematical theory of probability with a model of a random mechanism. In the sciences, probabilities are never made up or 'elicited' by observing the choices people make, or the bets they are willing to place.  The reason is that, in science and technology, interpreted probability exactifies objective chance, not gut feeling or intuition. No randomness, no probability." (Mario Bunge, "Chasing Reality: Strife over Realism", 2006)

"Most dashboards fail to communicate efficiently and effectively, not because of inadequate technology (at least not primarily), but because of poorly designed implementations. No matter how great the technology, a dashboard's success as a medium of communication is a product of design, a result of a display that speaks clearly and immediately. Dashboards can tap into the tremendous power of visual perception to communicate, but only if those who implement them understand visual perception and apply that understanding through design principles and practices that are aligned with the way people see and think." (Stephen Few, "Information Dashboard Design", 2006)

"The big part of the challenge is that data quality does not improve by itself or as a result of general IT advancements. Over the years, the onus of data quality improvement was placed on modern database technologies and better information systems. [...] In reality, most IT processes affect data quality negatively, Thus, if we do nothing, data quality will continuously deteriorate to the point where the data will become a huge liability." (Arkady Maydanchik, "Data Quality Assessment", 2007)

"The corporate data universe consists of numerous databases linked by countless real-time and batch data feeds. The data continuously move about and change. The databases are endlessly redesigned and upgraded, as are the programs responsible for data exchange. The typical result of this dynamic is that information systems get better, while data deteriorates. This is very unfortunate since it is the data quality that determines the intrinsic value of the data to the business and consumers. Information technology serves only as a magnifier for this intrinsic value. Thus, high quality data combined with effective technology is a great asset, but poor quality data combined with effective technology is an equally great liability." (Arkady Maydanchik, "Data Quality Assessment", 2007)

"Enterprise architecture is the process of translating business vision and strategy into effective enterprise change by creating, communicating and improving the key requirements, principles and models that describe the enterprise's future state and enable its evolution. The scope of the enterprise architecture includes the people, processes, information and technology of the enterprise, and their relationships to one another and to the external environment. Enterprise architects compose holistic solutions that address the business challenges of the enterprise and support the governance needed to implement them." (Anne Lapkin et al, "Gartner Clarifies the Definition of the Term 'Enterprise Architecture", 2008)

"Synergy occurs when organizational parts interact to produce a joint effect that is greater than the sum of the parts acting alone. As a result the organization may attain a special advantage with respect to cost, market power, technology, or employee." (Richard L Daft, "The Leadership Experience" 4th Ed., 2008)

"The butterfly effect demonstrates that complex dynamical systems are highly responsive and interconnected webs of feedback loops. It reminds us that we live in a highly interconnected world. Thus our actions within an organization can lead to a range of unpredicted responses and unexpected outcomes. This seriously calls into doubt the wisdom of believing that a major organizational change intervention will necessarily achieve its pre-planned and highly desired outcomes. Small changes in the social, technological, political, ecological or economic conditions can have major implications over time for organizations, communities, societies and even nations." (Elizabeth McMillan, "Complexity, Management and the Dynamics of Change: Challenges for practice", 2008)

"What’s next for technology and design? A lot less thinking about technology for technology’s sake, and a lot more thinking about design. Art humanizes technology and makes it understandable. Design is needed to make sense of information overload. It is why art and design will rise in importance during this century as we try to make sense of all the possibilities that digital technology now affords." (John Maeda, "Why Apple Leads the Way in Design", 2010) 

"Enterprise Architecture presently appears to be a grossly misunderstood concept among management. It is NOT an Information Technology issue. It is an ENTERPRISE issue. It is likely perceived to be an Information Technology issue as opposed to a Management issue for two reasons: (1) Awareness of it tends to surface in the Enterprise through the Information Systems community. (2) Information Technology people seem to have the skills to do Enterprise Architecture if any Enterprise Architecture is being or is to be done." (John A Zachman, 2011)

"Today, technology has lowered the barrier for others to share their opinion about what we should be focusing on. It is not just information overload; it is opinion overload." (Greg McKeown, "Essentialism: The Disciplined Pursuit of Less", 2014)

"We have let ourselves become enchanted by big data only because we exoticize technology. We’re impressed with small feats accomplished by computers alone, but we ignore big achievements from complementarity because the human contribution makes them less uncanny. Watson, Deep Blue, and ever-better machine learning algorithms are cool. But the most valuable companies in the future won’t ask what problems can be solved with computers alone. Instead, they’ll ask: how can computers help humans solve hard problems?" (Peter Thiel & Blake Masters, "Zero to One: Notes on Startups, or How to Build the Future", 2014)

"Technological change is discontinuous and difficult. It is a radical change in that it forces people to deal with the world in a different way, that is, it changes the world of experience." (William Byers, "Deep Thinking: What Mathematics Can Teach Us About the Mind", 2015)

"The problem with artificial intelligence and information technology is that they promise a methodology that would lead to a way of solving all problems - a self-generating technology that would apply to all situations without the need for new human insights and leaps of creativity." (William Byers, "Deep Thinking: What Mathematics Can Teach Us About the Mind", 2015)

"Technology systems are difficult to wrangle. Our systems grow in accidental complexity and complication over time. Sometimes we can succumb to thinking that other people really hold the cards, that they have the puppet strings we don’t." (Eben Hewitt, "Technology Strategy Patterns: Architecture as strategy" 2nd Ed., 2019)

"Technology is not a magic pill that can solve inadequacies in processes." (Jared Lane, "Why Companies Should Stop Making Digital Transformation A Science Project", 2021) [source]

"Always remember what you originally wanted the system to accomplish. Having the latest, greatest system and a flashy data center to boot is not what data processing is supposed to be all about. It is supposed to help the bottom line, not hinder it." (Richard S Rubin)

"The first rule of any technology used in a business is that automation applied to an efficient operation will magnify the efficiency. The second is that automation applied to an inefficient operation will magnify the inefficiency." (Bill Gates)