A Software Engineer and data professional's blog on Business Intelligence /Analytics, SQL Server & other DBMS, DIKW Management, Software Engineering/Programming, Project Management, Dynamics 365 & other ERP systems, respectively other IT related topics.
Pages
- 🏠Home
- 🗃️Definitions
- 🔢SQL Server
- 🎞️SQL Server: VoD
- 🏭Fabric
- 🎞️Fabric: VoD
- ⚡Power BI
- 🎞️Power BI: VoD
- 📚Data
- 📚Engineering
- 📚Management
- 📚SQL Server
- 🎞️D365: VoD
- 📚Systems Thinking
- ✂...Quotes
- 🧾D365: GL
- 💸D365: AP
- 💰D365: AR
- 🏠D365: FA
- 👥D365: HR
- ⛓️D365: SCM
- 🔤Acronyms
- 🪢Experts
- 🔠Dataviz & BI
- 🔠D365
- 🔠Fabric
- 🔠Engineering
- 🔠Management
- 🔡Glossary
- 🌐Resources
- 🏺Dataviz
- 🗺️Social
- 📅Events
- ℹ️ About
22 September 2009
⌛SQL Reloaded: Incremental Update Technique(s) in Oracle
💫ERP Systems: Learning about Oracle APPS internals II
SELECT * FROM ALL_ALL_TABLES WHERE OWNER = :OWNER SELECT * FROM ALL_VIEWS WHERE OWNER = :OWNER
Getting the differences between the two sources it’s pretty easy:
SELECT A.* FROM ALL_ALL_TABLES A LEFT JOIN APPS.FND_TABLES B ON A.TABLE_NAME = B.TABLE_NAME WHERE A.OWNER = :OWNER AND B.TABLE_NAME IS NULL
If you are interested on Oracle internals you can find more goodies by querying the Oracle dictionary table:
SELECT * FROM DICT WHERE TABLE_NAME LIKE '%VIEW%'
One of such goodies is the view that stores the list of Materialized views:
SELECT * FROM DBA_MVIEWS
PS: I know that these are pretty basic stuff for APPS DBAs though not all of us discovered them from beginning.
15 September 2009
🛢DBMS: Performance (Definitions)
"The speed with which SQL Server processes queries and returns results." (Karen Paulsell et al, "Sybase SQL Server: Performance and Tuning Guide", 1996)
"Accomplishment of project expectations; ability to attain goals by using resources efficiently and effectively." (Timothy J Kloppenborg et al, "Project Leadership", 2003)
"A measure of the degree to which a system or component accomplishes designated functions within given constraints, such as accuracy, time, or resource usage." (Richard D Stutzke, "Estimating Software-Intensive Systems: Projects, Products, and Processes", 2005)
"Performance is a measure of how fast a database services applications, and ultimately end-users." (Gavin Powell, "Beginning Database Design", 2006)
"quantitative measures of the execution of some element or set of elements, such as average execution time or memory usage." (Bruce P Douglass, "Real-Time Agility", 2009)
"Key characteristic of a computer system upon which it is compared to other systems." (Max Domeika, "Software Development for Embedded Multi-core Systems", 2011)
"The quality of outputs, outcomes, benefits, and results achieved." (Paul C Dinsmore et al, "Enterprise Project Governance", 2012)
"The manner in which organizations achieve results (i.e., the way they behave and operate to effect actions), as well as the outputs and outcomes of these actions (i.e., the results they achieve)." (Sally-Anne Pitt, "Internal Audit Quality", 2014)
"1)Output/results (or measures of same) The relationship between targeted output and achieved output Output or results can, and must, be measured. 2)Work Performance. How well a person has done in producing the results (output) taking into consideration all relevant circumstances. How well has the person carried out the work of the role? Work performance cannot be measured; it must be judged by a manager based on how well the person has worked to achieve the assigned output, or result, in the situation in which the work was performed." (Catherine Burke et al, "Systems Leadership, 2nd Ed,", 2018)
"a measure of what is achieved or delivered" (ITIL)
"The degree to which a system or component accomplishes its designated functions within given constraints regarding processing time and throughput rate. (IEEE 610)
14 September 2009
🛢DBMS: Vertical Partitioning (Definitions)
"To segment a single table into multiple tables based on selected columns. Each of the multiple tables has the same number of rows but fewer columns. See also horizontal filtering; vertical filtering." (Microsoft Corporation, "SQL Server 7.0 System Administration Training Kit", 1999)
"Vertical partitioning segments a single table into multiple tables based on selected columns. Each of the multiple tables has the same number of rows but fewer columns." (Anthony Sequeira & Brian Alderman, "The SQL Server 2000 Book", 2003)
"The process of splitting a single table into multiple tables based on selected columns. Each of the multiple tables has the same number of rows but fewer columns." (SQL Server 2012 Glossary, "Microsoft", 2012)
"Vertical partitioning is a technique for improving database performance by storing columns of a relational table in separate, multiple partitions. This is especially helpful when only a subset of columns is read from a row of data and the columns are read from many rows." (Dan Sullivan, "NoSQL for Mere Mortals®", 2015)
"Storing a relation as two or more tables that are projections of the original relation to improve database performance." (Jan L Harrington, "Relational Database Design and Implementation" 3rd Ed., 2009)
🛢DBMS: Cost Based Optimizer (Definitions)
"Software in a relational database that tries to determine how to process the query by assigning estimated costs to various table lookup alternatives." (Ralph Kimball & Margy Ross, "The Data Warehouse Toolkit" 2nd Ed., 2002)
"A cost-based optimizer evaluates the cost in terms of the amount of computing resources used for execution plans. The optimizer analyzes the possible plans and then chooses for execution the access plan with the lowest estimated cost." (Microsoft Corporation, "Microsoft SQL Server 7.0 Data Warehouse Training Kit", 2000)
"An Oracle optimizer methodology that relies on the characteristics of the tables being queried to determine the method used to run the query. A cost is calculated for estimated CPU, I/O, and sorting for the possible execution paths. The path with the lowest overall cost is used to perform the query." (Bob Bryla, "Oracle Database Foundations", 2004)
"A query optimizer technique that uses an algorithm based on statistics about the objects being accessed, that is, number of rows, indexes available, indexes sparsity, and so on." (Carlos Coronel et al, "Database Systems: Design, Implementation, and Management" 9th Ed., 2011)
"The legacy name for the optimizer. In earlier releases, the cost-based optimizer was an alternative to the rule-based optimizer (RBO)." (Oracle)
DBMS: Optimization (Definitions)
"The activity of choosing an efficient execution strategy for processing a query is called as Query optimization." (S. Sumathi & S. Esakkirajan, "Fundamentals of Relational Database Management Systems", 2007)
[automatic query optimization:] "A method by which a DBMS takes care of finding the most efficient access path for the execution of a query." (Carlos Coronel et al, "Database Systems: Design, Implementation, and Management 9th Ed", 2011)
[dynamic query optimization:] "Refers to the process of determining the SQL access strategy at run time, using the most up-to-date information about the database. Contrast with static query optimization." (Carlos Coronel et al, "Database Systems: Design, Implementation, and Management" 9th Ed, 2011)
[static query optimization:] "A query optimization mode in which the access path to a database is predetermined at compilation time. Contrast with dynamic query optimization." (Carlos Coronel et al, "Database Systems: Design, Implementation, and Management" 9th Ed., 2011)
"The overall process of choosing the most efficient means of executing a SQL statement." (Oracle, "Database SQL Tuning Guide Glossary", 2013)
[adaptive query optimization:] "A set of capabilities that enables the adaptive optimizer to make run-time adjustments to execution plans and discover additional information that can lead to better statistics. Adaptive optimization is helpful when existing statistics are not sufficient to generate an optimal plan." (Oracle, "Database SQL Tuning Guide Glossary", 2013)
"Query optimization determine the most efficient way to execute a given query by considering possible query plans. Indexes and materialized views are also part of this approach." (Piethein Strengholt, "Data Management at Scale", 2020)
13 September 2009
💎SQL Reloaded: Randomized Data Selection
12 September 2009
💎SQL Reloaded: Number of Records I (via COUNT)
-- general syntax SELECT count(*) NoRecords FROM <schema>.<table> -- example SELECT count(*) NoRecords FROM [Person].[Address]
Some prefer using a count(1) instead of a count(*), though it doesn't make a difference considering how SQL Server handles the two scripts in the background.
-- general syntax SELECT count(1) NoRecords FROM <schema>.<table> -- example SELECT count(1) NoRecords FROM [Person].[Address]
Same can be achieved by using a SUM:
-- general syntax SELECT SUM(1) NoRecords FROM <schema>.<table> -- example SELECT SUM(1) NoRecords FROM [Person].[Address]
However the SUM is typically used when counting only certain records within the same line (the attributes must be know however beforehand):
-- counting the occurrences for a certain values via SUM SELECT SUM(CASE WHEN Class IS NULL THEN 1 ELSE 0 END) [Nulls] , SUM(CASE WHEN Class = 'H' THEN 1 ELSE 0 END) [H] , SUM(CASE WHEN Class = 'L' THEN 1 ELSE 0 END) [L] , SUM(CASE WHEN Class = 'M' THEN 1 ELSE 0 END) [M] , count(*) NoRecords FROM [Production].[Product]
The sane output can be used via a GROUP BY based on the grouping attributes:
-- counting the occurrences for a certain attribute via COUNT SELECT Class , count(*) NoRecords FROM [Production].[Product] GROUP BY Class -- counting the occurrences for a group of attributes via COUNT SELECT Class , Color , count(*) NoRecords FROM [Production].[Product] GROUP BY Class , Color
If you want to implement the functionality in a function or stored procedure and provide the name of the table as parameter, then things get a little more complicated as the query needs to be built dynamically on runtime. In stored procedures a query built dynamically can be usually executed using EXEC or EXECUTE stored procedures, though none of them will do as we need to reuse the actual value returned by the query. This can be achieved with the help of a third stored procedure called sp_executesql, and the work can be done relatively easily with the below stored procedure:
-- dropping the stored procedure --DROP PROCEDURE IF EXISTS dbo.pGetNumberRecords -- creating the stored procedure CREATE PROCEDURE dbo.pGetNumberRecords( @Table varchar(100) , @NumberRecords int OUTPUT ) AS DECLARE @SglQuery nvarchar(250) IF IsNull(@table,'')<>'' BEGIN SET @SglQuery = N'SELECT @NumberRecords = count(*) FROM ' + @table --get the number of records EXEC sp_executesql @Query = @SglQuery , @params = N'@NumberRecords INT OUTPUT' , @NumberRecords = @NumberRecords OUTPUT END -- testing the stored procedure DECLARE @NumberRecords int EXEC dbo.pGetNumberRecords '[Production].[Product]', @NumberRecords OUTPUT SELECT @NumberRecords
The overhead of using a stored procedure for getting the records count is bearable though the solution is not necessarily elegant, a nicer approach being the use of functions for the same purpose:
-- dropping the function --DROP FUNCTION IF EXISTS dbo.GetNumberRecords -- creating the test function CREATE FUNCTION dbo.GetNumberRecords( @table as nvarchar(100)) RETURNS int AS BEGIN DECLARE @sql varchar(200) DECLARE @NumberRecords int IF IsNull(@table,'')<>'' BEGIN SET @sql = N'SELECT @NumberRecords = count(*) FROM ' + @table --get the number of records EXEC sp_executesql @Query = @sql , @params = N'@NumberRecords INT OUTPUT' , @NumberRecords = @NumberRecords OUTPUT END RETURN @NumberRecords END -- testing the function SELECT dbo.GetNumberRecords('[AdventureWorks2014].[Production].[Product]')
Unfortunately UDFs have the restriction that they can’t execute stored procedures, and thus the above technique can’t be used. Even if the function is created, when attempting to run it, the following error occurs:
Msg 557, Level 16, State 2, Line 185There is actually an alternative for executing pass-through queries in functions by using OPENQUERY, OPENROWSET or OPENDATASOURCE functions, allowing thus to execute DML statements against a given linked server, as is the case of first two functions, respectively using ad-hoc connection information as is the case of the third function. For exemplification I chose OPENQUERY function, this being the code created to return the number of records from a specified table (you'll need to replace
Only functions and some extended stored procedures can be executed from within a function.
--dropping the test function -- DROP FUNCTION IF EXISTS dbo.GetNumberRecordsTest -- creating the test function CREATE FUNCTION dbo.GetNumberRecordsTest() RETURNS int AS BEGIN RETURN ( SELECT NoRecords FROM OPENQUERY( <server_name>, 'SET NOCOUNT ON; SELECT count(*) NoRecords FROM [AdventureWorks2014].[Production].[Product]') AS O) END -- retrieving the number of records SELECT dbo.GetNumberRecordsTest()
For this piece of code to work is first necessary to refer the server as a linked server:
EXEC sp_serveroption <server_name>, 'Data Access', true
Withought doing this first you’ll get the error:
Msg 7411, Level 16, State 1, Procedure GetNumberRecords, Line 4 Server '>server_name<' is not configured for DATA ACCESS.You can use sp_helpserver to get server's name:
-- getting server's name EXEC sp_helpserver
The above function doesn’t help much. It would be helpful if we could provide the table’s name as parameter and store the query string in a variable, something like:
-- creating the test function CREATE FUNCTION dbo.GetNumberRecords( @TableName as nvarchar(100)) RETURNS int AS BEGIN DECLARE @sql varchar(200) SET @sql = 'SET NOCOUNT ON; SELECT count(*) NoRecords FROM ' + @TableName RETURN ( SELECT NoRecords FROM OPENQUERY(, @sql) AS O) END -- retrieving the number of records SELECT dbo.GetNumberRecords()
Unfortunately OPENQUERY function same as OPENROWSET and OPENDATASOURCE expect a constant parameter as query and therefore the error (as per SQL Server 2005):
Msg 102, Level 15, State 1, Procedure GetNumberRecords, Line 12 Incorrect syntax near '@sql'. Expecting STRING or TEXT_LEXI wonder why the SQL Server team hasn’t allowed the use of variables as parameters for OPENQUERY functions, if not in the previous SQL Server versions at least in SQL Server 2008! Anyway, another impediment would be raised by the deterministic nature of functions.
💎SQL Reloaded: Deterministic Functions (GetDate SQL Server 2000/2005)
-- creating the test function CREATE FUNCTION dbo.fGetCurrentDate() RETURNS smalldatetime AS BEGIN RETURN GetDate() END -- testing the function SELECT dbo.fGetCurrentDate()
Server: Msg 443, Level 16, State 1, Procedure GetCurrentDate, Line 6 Invalid use of 'getdate' within a function.
-- creating the test view CREATE VIEW dbo.vGetCurrentDate AS SELECT GetDate() AS CurrentDate
-- testing the view SELECT * FROM dbo.vGetCurrentDate
-- creating the test UDF CREATE FUNCTION dbo.udfFilterByDate( @date datetime) RETURNS TABLE AS RETURN( SELECT * FROM [AdventureWorks2014].Production.Product WHERE [SellStartDate]>=@Date) -- testing the UDF SELECT * FROM dbo.udfFilterByDate(GetDate()) -- testing the UDF (alternative) DECLARE @date as datetime SET @date = GetDate() SELECT * FROM dbo.udfFilterByDate(@date)
05 September 2009
🛢DBMS: Method (Definitions)
"The equivalent to a relational database stored procedure, except that it executes on the data contents of an object, within the bounds of that object." (Gavin Powell, "Beginning Database Design", 2006)
"Within a database, a stored procedure, stored function, or trigger." (Pramod J Sadalage & Scott W Ambler, "Refactoring Databases: Evolutionary Database Design", 2006)
"A program module that is part of a user-defined data type that is used to create objects." (Jan L Harrington, "SQL Clearly Explained" 3rd Ed., 2010)
"A database object that encapsulates procedural logic to provide behavior for a structured type. A method can be implemented as an SQL method or an external method. See also external method, routine, SQL method." (Sybase, "Open Server Server-Library/C Reference Manual", 2019)
"A method that is implemented entirely by using a subset of SQL statements and SQL PL statements. See also method, routine." (Sybase, "Open Server Server-Library/C Reference Manual", 2019)
🛢DBMS: Catalog View (Definitions)
"Contains metadata about objects found in a SQL Server instance." (Sara Morganand & Tobias Thernstrom , "MCITP Self-Paced Training Kit : Designing and Optimizing Data Access by Using Microsoft SQL Server 2005 - Exam 70-442", 2007)
"A catalog view returns a SQL Server database and server-specific metadata." (Michael Coles, "Pro T-SQL 2008 Programmer's Guide", 2008)
"View that returns a SQL Server database and server-specific metadata." (Miguel Cebollero et al, "Pro T-SQL Programmer’s Guide 4th Ed", 2015)
"In DB2 for Linux, UNIX, and Windows, a SYSCAT or SYSSTAT view on the catalog table. One of a set of views automatically created when a database is created. Catalog views contain information about the database and the objects in that database. Examples of information about the database are definitions of database objects and information about the authority that users have on these objects. See also catalog table." (Sybase, "Open Server Server-Library/C Reference Manual", 2019)
"Built-in views that form the system catalog for SQL Server." (Microsoft, 'SQL Server 2012 Glossary", 2012)
04 September 2009
🛢DBMS: Aggregate Query (Definitions)
"A query that summarizes information from multiple rows by including an aggregate function such as SUM or AVG. Aggregate queries that use the GROUP BY clause can also display subtotal information by creating groups of rows that have data in common." (Microsoft Corporation, "SQL Server 7.0 System Administration Training Kit", 1999)
"A T-SQL query that summarizes information from multiple rows by including an aggregate function, such as SUM or AVG." (Anthony Sequeira & Brian Alderman, "The SQL Server 2000 Book", 2003)
"A query using a GROUP BY clause to create a summary set of records (smaller number of records)." (Gavin Powell, "Beginning Database Design", 2006)
[grouping query:] "A query that groups rows of data based on common values in one or more columns and that optionally computes summary values from each group." (Jan L Harrington, "SQL Clearly Explained" 3rd Ed., 2010)
"A query (SQL statement) that summarizes information from multiple rows by including an aggregate function such as Sum or Avg." (Microsoft, "SQL Server 2012 Glossary", 2012)
About Me
- Adrian
- Koeln, NRW, Germany
- IT Professional with more than 25 years experience in IT in the area of full life-cycle of Web/Desktop/Database Applications Development, Software Engineering, Consultancy, Data Management, Data Quality, Data Migrations, Reporting, ERP implementations & support, Team/Project/IT Management, etc.