All about JSON Scenarios and value proposition for JSON data enabled in Azure SQL Database and SQL Server Ralph Kemperdick Digital Business Architect, Data Microsoft Deutschland GmbH

Our Sponsors If you think, that a SQL Saturday is a nice possibility to learn from and network with fellow SQL Server enthusiasts FOR FREE, I just ask you one thing: Visit the sponsor booths and chat with the sponsors! They are covering the expenses for each and every of you.

Very warm Welcome in the JSON HOME

Agenda Use cases and Scenarios Loading JSON data into the database Transforming Data with the database Servicing JSON to the application layer Advanced scenarios when working with JSON

Use cases Telemetry/sensor data from devices that generate different information in messages. Typically real-time analysis of telemetry data as the system should return immediate alerts or suggestions. Applicable for gaming, social media or any other system generating widely variable data. Product catalog with different product types and each type has different fields. Common fields are stored as regular columns and JSON is used for variable parts. Web/mobile data exchange where web client requests data as JSON via AJAX, NetBeans, etc. Schema de-normalization where tags and hashes serialized as JSON.

JSON Operations Load Transform Service/Utility Other

Data Loading: Log analysis 4/22/2018 Data Loading: Log analysis UTF-8 SELECT c.CountryRegion, min_income, COUNT(*) as Anzahl FROM OPENROWSET (BULK N'C:\demo\JSON\customers.csv' ,FORMATFILE = 'C:\demo\JSON\customers.xml' ,CODEPAGE = '65001') as c CROSS APPLY OPENJSON(IncomeBracket,'$.IncomeBracket') WITH (CountryRegion varchar(50) , min_income varchar(20)) WHERE c.customer_id >= 500 GROUP BY c.CountryRegion, min_income ORDER BY Anzahl desc, min_income, c.CountryRegion Read JSON Data (Path) © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Transform: Generate JSON Document 4/22/2018 Transform: Generate JSON Document SELECT c.CountryRegion, min_income, COUNT(*) as Anzahl FROM OPENROWSET (BULK N'C:\demo\JSON\customers.csv' ,FORMATFILE = 'C:\demo\JSON\customers.xml' ,CODEPAGE = '65001') as c CROSS APPLY OPENJSON(IncomeBracket,'$.IncomeBracket') WITH (CountryRegion varchar(50) , min_income varchar(20)) WHERE c.customer_id >= 500 GROUP BY c.CountryRegion, min_income ORDER BY Anzahl desc, min_income, c.CountryRegion FOR JSON AUTO AUTO The AUTO mode generates nesting in the resulting JSON by using heuristics based on the way the SELECT statement is specified. You have minimal control over the shape of the JSON generated. The nested FOR JSON queries can be written to generate JSON hierarchy beyond the JSON shape that is generated by AUTO mode heuristics. PATH                                                    The PATH mode allows control over the shape of the JSON. It requires a specific format for the resulting rowset that is generated because of query execution. This rowset format is then mapped into JSON shape. The power of PATH mode is to create wrappers and nested complex properties. Use names of the columns to determine how properties will be nested. Columns may contain dot-separated strings (e.g. info.address.town) and FOR JSON will use properties in this path to nest value of the column. ROOT [('RootName')] Specifies that a single, top-level element should be added to the resulting JSON object. You can optionally specify the root element name to generate. The default value is “root”. , -: ROOT('root') INCLUDE_NULL_VALUES By default, if a value of some cell in the result set is NULL, FOR JSON will not output this property in a JSON object. INCLUDE_NULL_VALUES option forces the engine to generate property even if it has NULL value. ESCAPING Content provided by FOR JSON query should be formatted according to the JSON specification. Special characters and their representation are defined in ECMA-404 The JSON Data Interchange Format specification. As an example, we are escaping double quotes (“) or backslashes (\) with backslash (i.e. \”, \\), converting new lines, tabs, and carriage returns to \n, \t, and \c, etc. Not supported is RAW, ELEMENTS, EXPLICIT, and TYPE options, since that AUTO and PATH cover most of the requirements. By default, cells with NULL values will not be generated in the output string. If they should be generated, use option INCLUDE_NULL_VALUES to force generation of properties without data. FOR JSON clause can be used in batches, stored procedures, user defined functions and triggers. Source of data can be table (disk-based or in-memory Hekaton table), view or synonym. © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

JSON Utility JSON_VALUE() JSON_QUERY() JSON_MODIFY() ISJSON()

Utility: Generate JSON Document 4/22/2018 Utility: Generate JSON Document SELECT c.CountryRegion, min_income, COUNT(*) as Anzahl FROM OPENROWSET (BULK N'C:\demo\JSON\customers.csv' ,FORMATFILE = 'C:\demo\JSON\customers.xml' ,CODEPAGE = '65001') as c CROSS APPLY OPENJSON(IncomeBracket,'$.IncomeBracket') WITH (CountryRegion varchar(50) , min_income varchar(20)) WHERE c.customer_id >= 500 GROUP BY c.CountryRegion, min_income ORDER BY Anzahl desc, min_income, c.CountryRegion FOR JSON AUTO AUTO The AUTO mode generates nesting in the resulting JSON by using heuristics based on the way the SELECT statement is specified. You have minimal control over the shape of the JSON generated. The nested FOR JSON queries can be written to generate JSON hierarchy beyond the JSON shape that is generated by AUTO mode heuristics. PATH                                                    The PATH mode allows control over the shape of the JSON. It requires a specific format for the resulting rowset that is generated because of query execution. This rowset format is then mapped into JSON shape. The power of PATH mode is to create wrappers and nested complex properties. Use names of the columns to determine how properties will be nested. Columns may contain dot-separated strings (e.g. info.address.town) and FOR JSON will use properties in this path to nest value of the column. ROOT [('RootName')] Specifies that a single, top-level element should be added to the resulting JSON object. You can optionally specify the root element name to generate. The default value is “root”. , -: ROOT('root') INCLUDE_NULL_VALUES By default, if a value of some cell in the result set is NULL, FOR JSON will not output this property in a JSON object. INCLUDE_NULL_VALUES option forces the engine to generate property even if it has NULL value. ESCAPING Content provided by FOR JSON query should be formatted according to the JSON specification. Special characters and their representation are defined in ECMA-404 The JSON Data Interchange Format specification. As an example, we are escaping double quotes (“) or backslashes (\) with backslash (i.e. \”, \\), converting new lines, tabs, and carriage returns to \n, \t, and \c, etc. Not supported is RAW, ELEMENTS, EXPLICIT, and TYPE options, since that AUTO and PATH cover most of the requirements. By default, cells with NULL values will not be generated in the output string. If they should be generated, use option INCLUDE_NULL_VALUES to force generation of properties without data. FOR JSON clause can be used in batches, stored procedures, user defined functions and triggers. Source of data can be table (disk-based or in-memory Hekaton table), view or synonym. © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Performance Text processing Indexes In-Memory OLTP 4/22/2018 Performance Text processing Indexes NONCLUSTERED indexes on computed columns Full text indexes on arrays or (key,value) pairs In-Memory OLTP Memory optimized, lock free, optimistic concurrency Native compilation Clustered Column Store Integration NVARCHAR(MAX) support 10-30x compression, batch mode processing Function Execution time (microsec) @xml.value() 30-50 JSON_VALUE 11-13 CHARINDEX 7-9 PATINDEX SUBSTRING 1-2 SUBSTRING+CHARINDEX 18-22 © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

NCI*: Index JSON path SELECT ProductID, Name FROM ProductCatalog 4/22/2018 NCI*: Index JSON path SELECT ProductID, Name FROM ProductCatalog WHERE JSON_VALUE(Data,'$.Color') = 'Silver' ALTER TABLE ProductCatalog ADD vColor AS JSON_VALUE(Data,'$.Color') CREATE INDEX idx_Color ON ProductCatalog(vColor) *NCI = Non Clustered Index © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

FTS*: Index any array elements 4/22/2018 FTS*: Index any array elements SELECT ProductID, Name FROM ProductCatalog WHERE CONTAINS(Keywords,'Silver OR Red') *FTS = Full Text Search © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

FTS*: Index any key:value 4/22/2018 FTS*: Index any key:value SELECT ProductID, Name FROM ProductCatalog WHERE CONTAINS(Data, '(Color~Silver) AND NEAR((Count,4),1)') AND JSON_VALUE(Data,'$.Color') = 'Silver' AND JSON_VALUE(Data,'$.Count') = '4' *FTS = Full Text Search © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

User defined Compression 4/22/2018 User defined Compression CREATE TABLE IF EXISTS Person CREATE TABLE Person( _id int identity primary key, data varbinary(max), value AS CAST(DECOMPRESS(data) AS nvarchar(max)) ) INSERT INTO Person(data) VALUES (COMPRESS(@json)) SELECT COMPRESS((SELECT * FROM tab FOR JSON PATH)) © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

New features Better string processing User defined compression 4/22/2018 New features Better string processing STRING_SPLIT FORMATMESSAGE User defined compression COMPRESS/DECOMPRESS Standard GZIP compression BULK IMPORT with UTF8 CODEPAGE = '65001' Language enhancements DROP IF EXISTS AT TIME ZONE DATEDIFF_BIG © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Q&A The Effect is called?

Save the Dates! PASS Camp 2016 - 06. to 09. December 2016 SQL Konferenz 2017 - 14. to 16. February 2017

How did you like it? Please give feedback to the event: http://goo.gl/5tStsd to me as a speaker: http://goo.gl/TgViTX