1. Welcome! Install MarkLogic Geophoto Install Node.js v4, NPM
git clone
Install git client
cd geophoto
Hands-on Node
npm install -g bower
git clone  on-node
npm install && bower install
cd ml-setup
cd hands-on-node
node setup.js bootstrap
npm install
cd ..
cd import
node import.js ../data/photos

2. MarkLogic: Hands-on with Node.js
David
Picture this… your boss walks in and says, "Great news! You know that big project we've heard so much about? You've been chosen to lead it!"

3. Agenda MarkLogic Demo: Geophoto Feature walk-through
Hands-on with Node

4. MarkLogic / Enterprise NoSQL Database Platform
POWERFUL
AGILE
TRUSTED
Native JSON
Store
Native XML Store
Cloud Ready (AWS)
Performance at scale
LDAP and Kerberos
Security
Scalable and Elastic
Native RDF Triple Store
Geospatial Support
Hadoop and HDFS
REST API
Security Certifications
Monitoring and Management
SQL Support
Multi-OS Support
Configuration Management
24/7
Engineering
Support
Full-text Search
Flexible Indexes
“New” badge – indicates features that are new in MarkLogic 8. MarkLogic 8 also includes enhancements to other features such as the REST API, Java Client API, and Incremental/Customizable backup. These features are currently available in the Early Access program and are not discussed in detail in this deck. They are only highlighted here for awareness. All of the other features on this slide are fully available in MarkLogic 7.
Powerful - Deliver more value, build better apps **these are all of MarkLogic’s unique features**
MarkLogic is designed for today’s data, helping you find answers in documents, relationships, and metadata by storing and managing JSON, XML, RDF, Geospatial data, and more. MarkLogic serves as an intelligent data layer, giving you the freedom to do more.
Agile - Prepare for and respond to change **these are all of the features that focus on ease-of-use and flexibility**
Enjoy the flexibility of NoSQL to integrate data, and deploy in any environment—whether using Amazon Web Services, virtual machines, or on-premise hardware. With the agility and adaptability of MarkLogic, you can build applications fast.
Trusted - Enterprise-ready for mission-critical uses **these are all the features that ensure MarkLogic meets enterprise requirements**
MarkLogic is a hardened platform that is trusted to run mission-critical applications. It has higher security certifications than any other NoSQL database, and has uncompromised data resiliency with features that ensure you will never lose data.
Schema Agnostic
Bitemporal
Real-time Alerting
Samplestack
ACID Transactions
Flexible Replication
Semantic Inference
Tiered Storage
MarkLogic Content Pump
XA Transactions
Customizable Backup
Customizable Failover
Server-side JavaScript
Fully Transactional
Ad-hoc Queries
Index Across Data Types
Point-in-time Recovery
Atomic Forests

5. Enterprise NoSQL Database Platform
Flexible Data Model
Search and
Query
Semantics
Scalability and Elasticity
ACID Transactions
Certified Security
Hadoop Integration
Store and manage JSON, XML, RDF, and Geospatial data with a document- centric, schema- agnostic database
Lightning fast, sophisticated, sub-second search and query across all of your data
Store and query linked data as RDF and SPARQL
Scale to petabytes of data without over- provisioning or over-spending
Avoid data loss, data corruption, and stale reads—even at speed and scale
Government- grade, granular, role-based security
Make your Hadoop better by connecting it to MarkLogic
These are the key features to focus on when learning about MarkLogic.
(click) Today, we'll focus primarily on the four areas. If want to go deeper in any of these, we'd be happy to discuss or to point you toward additional resources.
The previous slide showed ALL of the features that MarkLogic includes, but here we are focusing on the top 7 key features to help explain what MarkLogic is, and what makes the technology so unique and powerful. There is no other database in the world that has this list of features. To start, if you only know 2 things about MarkLogic, it’s the flexible data model and search and query. These two features are core to how MarkLogic works, and underpin a lot of the other features such as MarkLogic’s ability to scale while still maintaining complex and consistent transactions.
In MarkLogic 7 we introduced semantics. MarkLogic is a native document store, and also a native triple store. Triples are stored as RDF and queried with SPARQL—formats defined as W3C standard for linked data. With semantics, you can store and query billions of facts and relationships, and even infer new facts. These facts and relationships provide context for better search and provide flexible data modeling to integrate and link data from different sources.
Scalability and elasticity, ACID transactions, and security are three of MarkLogic’s key “enterprise” features to ensure you can easily store and manage all of your data while not breaking the bank, losing any data, or allowing data to get into the wrong hands. It turns out that these features are not to be taken for granted, because they are really hard to do right. MarkLogic has spent a decade building a hardened, trusted platform, and these features are some of the reasons why MarkLogic is the leading enterprise NoSQL database.
Lastly, MarkLogic integrates easily with Hadoop and will make Hadoop better. Hadoop has gotten traction lately but most people realize now that it’s not a database. It’s a great place to put your data, and MarkLogic has a lot of unique ways for doing more with your data if you currently have it in Hadoop.

6. Flexible Data Model
Store and manage JSON, XML, RDF, and Geospatial data with a document-centric, schema-agnostic database
JSON, XML, RDF, Geospatial data, and also large binaries—all stored and managed on a single unified platform
Document-centric and schema-agnostic for agility, reducing lost fidelity and functionality from data conversion and brittle ETL
Use the data format that makes the most sense, keeping the data in its most readable form
Short Description:
MarkLogic is designed for today’s data, including documents, relationships, and metadata. MarkLogic is the only database that can natively store and rapidly query XML, JSON, RDF, and geospatial data—providing a single powerful platform for all of your data. Not only that, but MarkLogic is remarkably easy to use, giving developers an agile database that adapts to change.
**Note: MarkLogic 7 can handle JSON data, but MarkLogic 8 will include the ability to natively store JSON data.
Longer Description:
MarkLogic is the only database that is optimized to store JSON, XML, RDF, and Geospatial data. MarkLogic can ingest the two most popular forms of document data—JSON and XML—natively, meaning there is no conversion required and no valuable data lost. MarkLogic can also ingest other sorts of data, from RDF relationships to text, geospatial data, binary video files, and PDFs—without the need for conversion. MarkLogic users start with more answers available because they start with better data.
More Background on Document Databases The term “NoSQL” is a broad descriptor covering a wide range of new databases, generally broken down into four main categories: document, key-value, column-family, and graph databases. Sometimes called “document stores” or “aggregate databases,” document databases use documents as the central entity for storage and queries. The term “document” does not necessarily mean a PDF or Microsoft Word document. The document can also be a single block of XML or JSON.
MarkLogic stores data as documents. Of the various types of NoSQL databases, document databases are the most popular. The document model makes it easier to understand what the data is about from a human perspective, and reduces the transformation required when moving data between tiers. This reduces the workload on the server and makes development much smoother.
Documents do not require a pre-defined schema before loading them into MarkLogic. Unlike with relational databases, you can change the data without mapping it to a fixed schema or hiding data in opaque objects. You can still store all of the information that you would find in the row of a relational table, but because it is stored and indexed as documents, you don’t have to normalize the data, and you don’t have to worry about how the shape of the data changes over time. This means you save enormous amounts of time and energy that would ordinarily be invested in ETL processes, and you also gain agility later on with future development. And, a unique feature of MarkLogic is that one document can contain other data types.

7. Flexible Data Model Schema-agnostic, structure-aware
<report>
Unstructured full-text
<title>
Suspicious vehicle near airport
Suspicious vehicle…
</title>
<date> Z
</date>
<type>
observation/surveillance
</type>
<threat>
<type>
suspicious activity
</type>
<category>
suspicious vehicle
</category>
Values
</threat>
XML, JSON – relational data often is hierarchical
transform data as needed – build your application sooner/faster
With MarkLogic, you can load all of your data as-is and only define a schema when you need it. You can even change your schema without having to redefine your entire data model. MarkLogic is also structure-aware, and you can even query the structure of documents. In MarkLogic, data is stored as self-contained documents – not in rows and columns – which means no foreign keys and no normalization. The data doesn’t have to be shredded across tables. Also, data is often in a document format already, such as XML, SGML, FpML, HTML, and JSON. When handling a document, MarkLogic starts by parsing and indexing the document contents, converting the document from serialized document format to a compressed binary fragment representation. Due to highly efficient compression, the data is much smaller than you would find with a typical file.
The example above shows how MarkLogic ‘sees’ an XML document in its hierarchical tree structure. Shown like this, you can see how the document model is self describing. This example shows a “Suspicious Activities Report”, but you could easily imagine how it could also be a trade document, medical record, book chapter, , metadata file—hundreds of different things that model well in a document structure. The example above shows something else that’s unique about MarkLogic as well. It shows various types of data including values, geospatial, unstructured full text, and semantic triples. All of this is indexed and can be queried.
More Information on Schemas
A database schema is a blueprint, or set of constraints, that define how data is structured and organized in the database. In the relational world, the schema is defined before ingesting data, and it has relations, tuples, and attributes represented as tables, rows, and columns. In the non-relational world, the relational mathematics at work with SQL do not apply, and schema is less rigid and does not have to be pre-defined. Well-formed XML, for example, can be parsed at ingestion and the database will use the inherent XML structure as the schema.
<location>
<lat>
</lat>
Semantic
(RDF) Triples
<long>
</long>
</location>
<description>
A blue van with license plate ABC 123 was observed parked behind the airport sign…
A blue van…
Geospatial
<triple>
<subject>
IRIID
</subject>
<predicate>
isa
</predicate>
<object>
license-plate
</object>
</triple>
<triple>
<subject>
IRIID
</subject>
<predicate>
value
</predicate>
<object>
ABC 123
</object>
</triple>
</description>

8. Search and Query
Built-in search to find answers in documents, relationships, and metadata
In MarkLogic, a search is a query, and a query is search
Ingest your data as-is and rely on over 30 sophisticated indexes to get better answers from today’s data
Lightning fast, sub-second search across hundreds of terabytes of data and billions of documents
Powerful, agile development providing complex query capability across heterogeneous data
Full-featured UX with full-text search, type-ahead suggestions, facets, snippeting, highlighted search terms, proximity boosting, relevance ranking, and language support
JavaScript
XQuery
SPARQL
Full-text Search
Rich Query Capability
In-database
MapReduce
Geospatial Search
Semantic Search
Short Description:
MarkLogic has built-in search and query capabilities. MarkLogic’s sophisticated indexes provide the power to search and query across hundreds of terabytes worth of documents, relationships, and metadata with the flexibility of multiple query languages.
*Note: Server-side JavaScript is a MarkLogic 8 feature.
Longer Description:
Most databases separate search and query into two distinct functions. MarkLogic changes that, starting with the idea that you should be able to ask your database what’s inside of it. This means not having to bolt-on a separate search solution, and not having to worry about when and how to build the right indexes, or how those indexes can be utilized to perform certain queries. MarkLogic is designed with over 30 sophisticated indexes that can be adjusted and tuned to make even the most complex queries as fast as possible without requiring data duplication, and data is ingested as-is and immediately searchable.
The sophisticated indexes mean that developers can ask harder questions and get faster responses. MarkLogic uses multiple query languages for each data types (JavaScript for JSON, XQuery for XML, and SPARQL for RDF). These query languages enable full-text search across unstructured content, rich query capability needed to make complex queries fast, Geospatial search for multiple formats and types (including connections to ESRI ArcGIS and Google Maps), Semantic search across linked data (similar to graph search, and MarkLogic 8 even includes inferencing), and also in-database MapReduce for running massive parallelized queries. One of the unique capabilities of MarkLogic is that the indexes are designed so that developers can write complex queries that run across multiple indexes without causing a performance bottleneck. With MarkLogic, you can query data as-is, or transform and manage data in-place—all with the reliability of a transactional system that maintains full ACID properties.
But, it’s important not to overlook the enterprise search experience. Many of MarkLogic’s first customers such as Elsevier were publishers who just needed a way to quickly search across massive amounts of content. The user experience is not too different from that of any major Web search engines, and in fact, MarkLogic’s founder Christopher Lindblad came from the search world, having been the architect on Ultraseek Server, an early enterprise search application developed at Infoseek. MarkLogic has many of the same features that user’s now expect in an enterprise search application, such as type-ahead suggestions, relevance ranking, and snippeting. MarkLogic also includes language support for over 200 languages, including advanced support with tokenization, stemming, and collation for some of the most common languages.
And, just to reiterate, all of this comes built-in with MarkLogic—you don’t have to bolt-on any other solution. This simplifies your architecture, and makes things incredibly easy for DBAs and developers. Having integrated search means one less additional platform to worry about. Developers don’t have to use a “lite” version of other search software during testing and eliminate additional, and unnecessary ETL procedures, which reduces risk. System-wide setting such as security are setup once and applied everywhere. If permissions are updated on documents, those updates are reflected automatically and immediately in searches.

9. Universal Index MarkLogic indexes…
Which vetted reports contain the phrase blue van?
Universal Index
Term
Term List
Document References
125, 516, 890, …
“blue” 123, 125, 129, 130, 152, 344, …
“van” 123, 125, 126, 129, 130, 152, …
“observed” 125, 152, 516, 522, 765, 890, …
MarkLogic indexes…
Words
Phrases
Stemmed words and phrases
Structure
Words and phrases in the context of structure
Values
Collections
Security Permissions
“blue van” 123, 125, 129, 130, 152, 486, …
STEM “observe” 125, 152, 516, 522, 765, 890, …
<report> …
<report>/<location> …
<threat>/<category> …
<type>suspicious activity</type> …
MarkLogic’s sophisticated indexes take a little bit more explaining, so these slides are provided to show an example of how the indexes work, using the example from the data on one of the previous slides.
The Universal Index is the main MarkLogic index, and is always turned on by default. It uses inverted indexes and term lists to label everything in the database—structured or unstructured. It’s this index that makes MarkLogic’s search and query functionality so powerful. The Universal Index is also useful for many other things, like collections, directories, and security rules. For more information on these indexes and how they work, read the white paper, Inside MarkLogic, that was written by MarkLogic’s Chief Architect, Jason Hunter.
<date> Z</date> …
Collection:Vetted …
Role:Analyst + Action:Read …
… …
… …
… …

10. Which vetted reports containing the phrase blue van were submitted before 2015?
Range Index
Term
Term List
Document References
125, 516, 890, …
“blue” 123, 125, 129, 130, 152, 344, …
“van” 123, 125, 126, 129, 130, 152, …
“observed” 125, 152, 516, 522, 765, 890, …
“blue van” 123, 125, 129, 130, 152, 486, …
STEM “observe” 125, 152, 516, 522, 765, 890, …
<report> …
<report>/<location> …
<threat>/<category> …
Range indexes map document IDs to values, and vice-versa in a compact in-memory representation.
<type>suspicious activity</type> …
One of the next important indexes we’ll need to use in our example is the Range Index. The range index is useful for a variety of things, including simply searching dates really quickly and returning the results or extracting information from the documents in the result set. For each range index, MarkLogic creates data structures that make it easy to do two things: for any fragment id get the fragment's range index value(s), or, for any range index value get the fragment ids that have that value. It works really fast and doesn’t waste memory or disk.
Also, as shown above, range queries can be combined with any other types of queries in MarkLogic. Say you want to limit results to those within a date range as above but also having a certain metadata tag. MarkLogic uses the range index to get the set of fragment ids in the range, uses a term list to get the set of fragment ids with the metadata tag, and intersects the sets of ids to determine the set of results matching both constraints. All indexes in MarkLogic are fully composable with each other.
In addition to organizing dates and documents IDs, the range index is also useful for joining documents, which is done by finding the ID’s for one set of documents, then feeding that into a second query as a constraint. And, it’s also good for sorting information, and is the index that enables facets—one of the key MarkLogic search features. For example, if we wanted to organize search results in categories and display the results down the left-hand side—similar to how Amazon shows products, but in MarkLogic you can build it all on the fly.
One other thing about the range index is that it’s also really fast without wasting any space because it is really implemented by two data structures, one written to disk and then another memory mapped for efficient access.
<date> Z</date> …
Collection:Vetted …
Role:Analyst + Action:Read …
… …
… …
… …

11. Triple Index Geospatial Index
Which vetted reports about a blue van from before 2015 in this location have a New England license plate?
Which vetted reports about a blue van from before 2015 refer to a location near the airport?
Geospatial Index
Triple Index
Term
Term List
Document References
125, 516, 890, …
“blue” 123, 125, 129, 130, 152, 344, …
“van” 123, 125, 126, 129, 130, 152, …
“observed” 125, 152, 516, 522, 765, 890, …
“blue van” 123, 125, 129, 130, 152, 486, …
STEM “observe” 125, 152, 516, 522, 765, 890, …
<report> …
<report>/<location> …
<threat>/<category> …
The Geospatial index is like a 2D range index, with built-in query support for point, box, circle, and complex polygons.
<type>suspicious activity</type> …
The Geospatial index is similar to a range index, with built-in support for point, box, circle, linestrings, and complex polygons. MarkLogic also supports multiple geospatial data types such as GML, KML, and GeoRSS. In the example above, the geospatial index is used in concert with the other indexes to further narrow down the result set.
If you need to visualize the data, you can generate maps that display information using multiple layers of data, and include pop-ups that let users drill further into the data and documents displayed. Users can conduct full text searches across millions of documents, and then filter and layer the data to create new insights. MarkLogic integrates with a variety of geographic aware products such as Esri ArcGIS, Google Earth, Google Maps, Yahoo Maps, and Microsoft Bing Maps.
Another geospatial capability is real-time alerting. It’s not highlighted in this example, but it’s a really useful feature if you want to push real-time alerts when new data is ingested that matches a saved query, providing a stream of information that can even be delivered to you by text or . When enabled, geospatial queries are indexed and new information is routed as soon as it arrives. The time it takes to check new information stays constant as the number of alerting queries increases, giving MarkLogic the ability to handle millions of different alerts simultaneously and still maintain high performance.
<date> Z</date> …
Collection:Vetted …
Role:Analyst + Action:Read …
… …
… …
… …

12. Triple Index All indexes fully composable for sub-second response to
Which vetted reports about a blue van from before 2015 in this location have a New England license plate?
Triple Index
“blue” 123, 125, 129, 130, 152, 344, …
“van” 123, 125, 126, 129, 130, 152, …
“observed” 125, 152, 516, 522, 765, 890, …
“blue van” 123, 125, 129, 130, 152, 486, …
STEM “observe” 125, 152, 516, 522, 765, 890, …
<report> …
<report>/<location> …
<threat>/<category> …
<type>suspicious activity</type> …
<date> Z</date> …
Collection:Vetted …
Role:Analyst + Action:Read …
… …
Term
Term List
Document References
125, 516, 890, …
All indexes fully composable
for sub-second response to
very complex queries
The Triple index is an index of “facts” expressed as Semantic triples. It can efficiently query and join billions of “linked data” triples.
In addition to the universal index, range index, and geospatial index, MarkLogic also has a triple index. This index is what powers the semantics capabilities for storing and managing RDF triples. Triples are facts—a subject, predicate, and object—that are stored natively in MarkLogic and can be queried with SPARQL. Both RDF and SPARQL are W3C standards for linked data.
In our example above, we have some facts from our data, such as “IRIID isa license plate” and “IRRID value ABC 123.” The triple index makes it possible for a query that is searching for “partial plate 123” to easily find that fact and relate it to the document in which that information is stored.
Semantics is discussed more in depth as its own unique feature, but it’s important to point out how semantics can be combined in combination queries like this across all of the data very efficiently.
To finish out the example, all of these indexes work together—they are fully composable—so that you can find better answer’s with today’s data. In this example, we quickly found the answer to “Which vetted reports about a blue van from before 2013 that refer to a location near an airport and refer to partial plate ABC.” The speed and efficiency of MarkLogic makes this possible—makes it possible to search and discover information in new ways. No other single database can do that.

13. ACID Transactions Don’t settle for a BASE-ic database
Reads and writes are durably logged to disk, and strongly isolated from other transactions
Prevents data corruption, stale reads, and inconsistent data—common problems with databases that settle for eventual consistency—and all of which are unacceptable
No performance drop-offs at scale. Production applications run tens of thousands of very complex transactions per second for tens of thousands of users
Accomplished using MVCC (multi-version concurrency control)
Short Description:
Don’t settle for a BASE-ic database. If you are running a transactional application, then you need all of the ACID properties so that reads and writes are durably logged to disk, and strongly isolated from other transactions. Without this feature, you run the risk of encountering data corruption, stale reads, and inconsistent data—all of which are unacceptable.
Longer Description:
If you are running a transactional application, then you need all of the ACID properties so that reads and writes are durably logged to disk, and strongly isolated from other transactions. Without this feature, you run the risk of encountering data corruption, stale reads, and inconsistent data—all of which are unacceptable. ACID stands for atomicity, concurrency, isolation, and durability.
MarkLogic gets ACID transactions by using MVCC (multi-version concurrency control). In an MVCC system, changes are tracked with a timestamp number which increments as transactions occur within a cluster. Each document gets its own creation timestamp and deletion timestamp (starting at infinity for documents not yet deleted). This design has many additional benefits beyond just ACID transactions: Very high throughput, read queries do not require locks, queries and updates don’t conflict, zero-latency indexing, point-in-time query and recovery, and fast database rollback.
Trusted Databases Have ACID Transactions
ACID transactions have been a standard for years, ensuring that organizations can trust their database to not lose data. For some applications, ACID support may be a nice-to-have, but it is an absolute necessity when it comes to mission-critical applications that involve financial data, healthcare data, or sensitive information pertaining to National security. But, it's also nice to have all of our tweets.
All of Marklogic's customers trust MarkLogic because it has ACID transactions and other enterprise features such as government-grade security and HA/DR. Below are some examples of the large enterprise organizations that rely on MarkLogic to store and manage their most critical data:
The Center for Medicare and Medicaid Services runs healthcare.gov on MarkLogic, achieving over 5,500 transactions per second to help over 8 million people sign up for health insurance
A Top-5 Investment Bank runs their operational trade store on MarkLogic, processing over 100,000 complex trades each day that typically result in about 32 million live deals in the system at any one time
A Large Energy Provider uses MarkLogic to gain a holistic view of their internal commercial trading operations by looking at various sources of data (trading data, market data, communication data, weather data, etc.)
A U.S. Combatant Command uses MarkLogic to collect over 100 million documents worth of Intelligence, Surveillance, and Reconnaissance (ISR) information
A Large Health Insurer uses MarkLogic to manage over 200 human resources-related data feeds, consisting primarily of complex heterogeneous structured data including payroll data, employee evaluations, promotions, benefit, etc.

14. ACID Transactions Implemented Using MVCC MVCC Benefits: 423 ∞ ∞ 628 ∞
/articles/doc1.xml
Document
Title
Author
Section
First
Last
Metadata
/articles/doc1.xml
Document
Title
Author
Section
First
Last
Metadata
MVCC Benefits:
ACID transactions
Zero-latency search indexing
High throughput
Lock-free reads
Serial writes
Point-in-time query
Fast database rollback
Year
ACID transactions are implemented using MVCC. Here is a detailed example of how MVCC works in MarkLogic, though MVCC is not unique to MarkLogic—it’s a well documented approach to achieving transactional consistency.
Benefits of MVCC
ACID transactions—ensure you don’t run the risk of encountering data corruption, stale reads, and inconsistent data
Better search and query with zero-latency search indexing, high throughput, point-in-time query, and lock-free reads
Fast database rollback that maintains full consistency in the case of disaster recovery
Script for Presenting
As mentioned previously, MarkLogic stores data as documents. Here’s an example of an XML document
[click]
In this example the document has the name “doc1”. In practice, these are actually fragments of documents, but here we’ll just say it’s the whole document. Initially, the document is given infinity timestamps as it goes through the process of parsing and indexing, and writing it to the in-memory stand
Each document has a creation timestamp and a deletion timestamp. On disk you'll see these timestamps in the Timestamps file.
When the document is ingested to MarkLogic, it is marked with a creation timestamp.
Here let’s say that was “423” and the deletion timestamp is just set to infinity.
For a request that doesn't modify data (called a query, as opposed to an update that might make changes), the system gets a performance boost by skipping the need for any document locking. The query is viewed as running at a certain timestamp, and throughout its life it sees a consistent view of the database at that timestamp, even as other (update) requests continue forward and change the data.
But, for this example, let’s say we want to move forward with an update transaction.
An update request uses use read/write locks to maintain system integrity while making changes. An update has to obtain a read-lock before reading a document and a write-lock before changing (adding, deleting, modifying) a document. Lock acquisition is ordered, first-come first-served, and a request waiting on a write-lock will block any newly requested read-locks for the same resource (otherwise it might never actually get the write-lock). Locks are released automatically at the end of a request.
If another write-lock were already present on the document from another executing update, the read-lock would have blocked until the other write-lock released. If another read-lock were already present, the lock promotion to a write-lock would have blocked.
So, after moving forward with the read/write locks, we can perform the transaction, but rather than updating the existing document, a new document is created [click]
This new document will have a new field for the date, here it’s shown by “year”
And, this new document will have timestamps of infinity initially
The new document is then given a new timestamp when it’s finished being created, and at that point the old document is given a deletion timestamp that matches the creation timestamp of the new document. This combination effectively replaces the old fragment with the new.
When the request concludes, it releases its locks. Our document is now deleted, replaced by a new shinier model.
The old document still exists on disk, of course. In fact, any query that was already in progress before the update incremented the timestamp, or any query doing time travel with an old timestamp, can still see it.
Also, note that if there were any problems during the transaction, the update exits with an error, and all pending changes that were part of that update are discarded.
423 ∞ ∞
628 ∞
628 ∞
Creation Timestamp
Deletion Timestamp

15. Government-Grade Security
Certified, granular security for modern data governance
Certified security – Higher security certifications than any other NoSQL database, carrying a Common Criteria Security Certification and certified to run in classified government systems
Granular Security – Role Based Access Control (RBAC) at the document level, and can also employ other models for cell-level security
Data Governance With MarkLogic
Privacy
Provenance
Compliance
Continuity
Retention
Security
Short Description:
When it comes to data governance, security is the most critical aspect. MarkLogic has focused on having government-grade security from the start, and has certified, granular security for modern data governance and to handle the increased complexity of today’s cyber threats.
Longer Description:
The risk of not securing data is simply too high, which is why, according to Gartner, investment in IT security will increase by around 39%, to $93 billion, by This is because the number, sophistication, and severity of cyber attacks continues to increase. In 2013, a single cyber incident cost U.S. companies $5.4M on average, or $188 per record. Organizations need to make security a top consideration when choosing their database, and MarkLogic has the government-grades security required to serve as the trusted platform to run their mission-critical applications, whether it’s at the DoD, FAA, CMS, the Intelligence Community, commercial healthcare organizations, or tier-1 investment banks. MarkLogic has focused on having government-grade security from the start, and has certified, granular security for modern data governance and to handle the increased complexity of today’s cyber threats.
MarkLogic is the only NoSQL database with NIAP Common Criteria certification, and MarkLogic is installed and operational on government systems that have gone through stringent security testing for handling classified data and are PL3/ICD 503 and DITSCAP Certified and Accredited (PL3 refers to the “protection level” that they system has achieved. It covers stringent measures for access, authentication, management, audits, role separation, and system assurance. PL3 was later superseded by ICD 503. DITSCAP is a DoD IT Security Capability and Accreditation Profile, similar in some respects to PL3 but more specific to the DoD than the IC. MarkLogic is installed and Operational on a government system that is DITSCAP Certified and Accredited. For detailed information on DCI Directive 6/3:
Beyond just having the right model and certifications, there are other features of MarkLogic that help more broadly with data governance. Security and privacy are the most important aspects of data governance, but it is also important to consider provenance (where data comes from), compliance (all the rules and regulations that industries must follow across the data lifecycle), continuity (risk management and disaster recovery), and retention (storage policies for handling data over time). Features such as tiered storage, bitemporal, and customizable backups all work in concert to ensure that data is cost-effectively and easily managed over the entire life-cycle.
How does security work in MarkLogic? …
Granular, Document-level Security
MarkLogic uses a document model to store each piece of data, and each document is governed by specific roles and permissions. This tight control is similar to cell-level security in key-value stores—much different from the database-level security in many other NoSQL databases.
Role-based, Document-based Security
MarkLogic uses a role-based access control (RBAC) security model in which each user is assigned any number of roles, and these roles are associated with any number of permissions and privileges. Permissions govern certain capabilities (read, insert, update) and privileges govern the creation of documents and execution of functions (URI and execute privileges). Security checks simply verify the necessary credentials before granting the requested action.
Additional Security Models
Beyond RBAC, MarkLogic also supports Attribute-Based Access Control (ABAC), Policy-Based Access Control (PBAC), and Label-Based Access Control (LBAC). These models further restrict access based on attributes (e.g., social security number, IP address, user’s age, or time of day), external policies or policy information stored in document metadata, or simple labels representing “high” or “low” levels of trust. This is how MarkLogic can get “cell level security.”
Authentication
To create secure HTTPS connections between browsers and app servers, MarkLogic uses a standard handshake procedure using SSL certificates. To further increase security, MarkLogic supports mutual authentication, so that both the client and the server are authenticated. MarkLogic also supports external authentication using either Lightweight Directory Access Protocol (LDAP) or Kerberos.
Auditing
MarkLogic includes an auditing capability to capture security-relevant events to monitor suspicious database activity or to satisfy applicable auditing requirements. Users can audit document access and updates, configuration changes, administrative actions, code execution, and changes to access control.
Security Database
A specific security database is created when MarkLogic is installed in order to store the user data, privilege data, role data, and other security information.
Compartment Security
MarkLogic provides compartment security for government customers, which delivers an even more granular level of control. With compartment security, a user must have all of the right roles rather than just one of the right roles to be granted access.

16. MarkLogic 8 / More Powerful, Easier to Use
Semantics
Developer Experience
MarkLogic 8 is more powerful than ever, but remarkably easy to use
Enterprise triple store, document store, database combined
Bitemporal
JSON
Unified indexing and query for today’s web and SOA data
Node.js Client API
Enterprise NoSQL database for Node.js applications
Java Client API
NoSQL agility in a pure Java interface
Server-side JavaScript
JavaScript runtime inside MarkLogic using Google’s V8
Note: This slide shows the key focus areas in MarkLogic. It doesn’t list all of the key features in MarkLogic 8. There are other slides in this deck that mention the other 4 major MarkLogic 8 features. They are Flexible Replication, Incremental Backup, Prescriptive Sizing, and an Enhanced HTTP Server.
Finger poll: 1-10, how comfortable are you with JavaScript? (10=I practically wrote the language; 1=that's how I order coffee, right?)
Paragraph Description of MarkLogic 8:
MarkLogic 8 is more powerful than ever, but remarkably easy to use. MarkLogic 8 now stores JSON natively in addition to XML and RDF. It also includes Server-side JavaScript, providing developers improved access to their data in the most popular programming language. But, MarkLogic 8 isn’t just a great developer experience, it also pushes the boundaries of what is expected of a database with new features including bitemporal and semantic inferencing. More powerful, agile, and trusted—MarkLogic 8 is the database for all of your data.
Three major things MarkLogic 8 is focused on:
One Powerful Database Platform Natively store and rapidly query JSON, XML, RDF, Geospatial data, and more—all in one unified platform and without the need for up-front data modeling.
Remarkably Easy to Use More control and access with all the right APIs, a Samplestack application that encapsulates best practices, and free training so developers and DBAs have everything they need right at their fingertips.
Smarter Data, Smarter Apps Real-time logic in the data layer with semantic inferencing, bitemporal, and alerting—these innovative features allow you to build smarter applications on a more adaptive platform.
Track information along two dimensions of time

17. JSON Unified indexing and query for today’s web and SOA data
Speed up development with powerful built-in search, transformation, and alerting capabilities designed for JSON
Reduce lost fidelity and functionality from data model translations and brittle ETL
Simplify architecture with data, metadata, and relationships managed consistently and securely together
Ease modern, end-to-end JavaScript development 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 {
"_id": 1,
"name": { "MarkLogic" },
"supports" : [
    {
    "datatype": "XML",
    "year": 2003
    },
    {
    "datatype": "JSON",
    "year": 2015
    } ] }
Short Description: Store and manage data natively as JSON documents, speeding up development and reducing data transformation with a simplified architecture for end-to-end JavaScript development.
Longer Description: Store JSON documents natively in MarkLogic. JSON storage brings all of MarkLogic’s production-proven indexing, data management, and security capabilities to the predominant data format of the Web. With JSON, you can build rich applications that span JSON, RDF, XML, binary, and text documents within a single environment without worrying about slow and brittle conversion between data formats. JSON storage complements the server-side JavaScript environment by providing seamless access to JSON managed in a MarkLogic cluster. Because native JSON uses the same foundation that MarkLogic has developed and tuned over the last decade, it is fast, stable, secure, and composable with existing data and code.
Script for Presenting: # Native JSON
> Unified indexing and query for today’s web and SOA data
JSON is everywhere. As a subset of JavaScript, it is the most convenient way to describe and share data structures with a web browser. Given the ubiquity of JSON and JavaScript in the browser—it’s the “X” in AJAX—JSON has emerged as *the* lightweight data format for sharing data structures. As a result of this momentum, JSON over HTTP is increasingly the go-to for web and back-office services too. Every language or environment has a JSON parser and an HTTP client. The ability to use JSON in the front-end—a web browser, a mobile app, or a service endpoint, the middle tier, *and* the database saves developers time.
> Speed up development with powerful built-in search, transformation, and alerting capabilities designed for JSON
Much of the work in developing an application with a relational database is the tricky modeling and plumbing work to map a *logical* model of the data, typically a graph of related objects in Java or C# or Ruby, to the physical model— tables and joins. JSON reduces this impedance mismatch by supporting structures that are more closely aligned with the logical model in your application code.
> Reduce lossy data model translations and brittle ETL
By using JSON, developers write less low-value mechanical data transformation code. Of course, the code you don’t write is the highest performance and least likely to cause bugs.
MarkLogic stores and manages JSON natively. This means you get all the advantages of an enterprise-grade database, using the data format of your applications without a bunch of brittle translation plumbing in between. MarkLogic indexes all of the values and structure of any JSON document without requiring up-front data modeling, like a relational database or legacy EDW. This means you can manage one or many different data models in the same database.
> Simplify architecture with data, metadata, and relationships managed consistently and securely together
Because Native JSON is built on MarkLogic’s production-proven foundation, your application can take advantage of the rich indexes, multi-document transactions, government-grade security, and high-performance compression that’s been built into MarkLogic for more than a decade. You can also build rich applications that combine data structures in JSON, marked-up text in XML, or facts and relationships in RDF triples. This gives you the ability to choose the best tool for the job—and the skills of your developers—without having to compromise on performance, reliability or the simplicity of your architecture.
> Ease modern, end-to-end JavaScript development
MarkLogic 8 also includes a full JavaScript runtime in the database, based on Google’s popular V8 engine. This allows you to write targeted logic to query or manage your JSON documents using MarkLogic’s rich set of APIs, in the language that is most natural. Moreover, deployed like a stored procedure across a MarkLogic cluster, server-side JavaScript runs close to the data for optimal performance.
# Better answers from today’s data
MarkLogic is designed for finding answers in varied, sometimes messy data, without having to understand every nuance of the structure or values ahead of time. We’ve chosen documents as the core representation of this data. Documents allow developers to model complex, often hierarchical, data in a format that likely looks something similar to how it’s exposed in a UI or even a physical artifact, like a contract or a record of a patient visit to a hospital. Increasingly, these documents are in the form of JSON.
MarkLogic’s unique approach to indexing documents allows you to take advantage of all of the aspects of rich, multi- structured data without having to first map it to a canonical “schema”, as you would in a relational data base. The rich out-of-the-box indexes and production-proven transactions and security mean you don’t have to give up reliability or performance to get the agility of NoSQL.
# Straightforward data integration
> Reduce the cost of translations and duplication working with data from multiple sources, likely already JSON
> React more quickly to change by using data in its natural format with less ETL
> Reduce risk and governance costs with fewer data silos
> Reuse existing JavaScript skills and tools for processing
———————————————————————————————
# Notes
* Not an either/or: Can manage XML and JSON together

18. Node.js Client API Enterprise NoSQL database for Node.js applications
Focus on application features rather than plumbing with out-of-the-box search, transactions, aggregates, alerting, geospatial, and more
Move faster to production with proven reliability at scale
Maximize performance and flexibility—bringing code to the data
Enable modern end-to-end JavaScript development
Always open source on GitHub
Participate. Contribute.
Fork it.
Short Description: A fluent JavaScript interface for key MarkLogic capabilities such as search, transactions, aggregates, alerting, and geospatial—the Node.js Client API allows developers to move faster to production with proven reliability at scale.
Longer Description: The MarkLogic Node.js Client API is an open source JavaScript library that allows developers to quickly, easily, and reliably access MarkLogic from their Node applications. You can create three-tiered application architectures using JavaScript and JSON throughout by leveraging the power of Node.js—the most popular web server for easily building fast, scalable network applications using non-blocking, asynchronous I/O. With Node.js in MarkLogic 8, you can get going with almost no up-front configuration because it includes a built-in database (Documents) and introduces a default REST server (port 8000), through which the Node.js and Java Client APIs communicate.
Script for Presenting: Short Description: A fluent JavaScript interface for key MarkLogic capabilities such as search, transactions, aggregates, alerting, and geospatial—the Node.js Client API allows developers to move faster to production with proven reliability at scale.
Script for Presenting: # Node.js Client API
> **Enterprise NoSQL database for Node.js applications**
Node.js has rapidly emerged as glue in the enterprise for building services with JavaScript. It provides a quick way to build and connect lightweight services with the language and tools that developers already know and love.
MarkLogic 8 is the best database to back these services when you
* Need to integrate heterogeneous data or
* Need something that your ops team can support in a production environment.
> **Focus on application features rather than plumbing: Out-of-the-box search, transactions, aggregates, alerting, geo**, …
One of the advantages of a database like MarkLogic is the amount of plumbing code that you *don’t* have to write in your Node.js application. Out-of-the-box MarkLogic gives you a rich set of indexes that are designed to be used together. For example, you can combine full-text, geospatial, and date range criteria in the same query. Save that query to the database and build real-time alerts.
The Node.js Client provides a fluent API that allows developers to easily integrate these capabilities into their applications. It operates in the “Node” way leveraging streams and promises and is fully asynchronous.
> **Move faster to production with proven reliability at scale**
Getting started with MarkLogic in a Node.js application is quick and easy. However, where MarkLogic really shines is when you need to move from a proof-of-concept to a production application. MarkLogic has more than a decade of experience being deployed in production across different industries. We’ve integrated best practices for performance and resource usage into the Node Client API. The code that you write in development will scale with your application as it moves to production.
> **Maximize performance, flexibility bringing code to the data**
For cases where you want to drop down to lower-level primitives to do something that the API doesn’t do out-of-the- box, MarkLogic 8 provides a full JavaScript runtime in the database. This allows you to package up code that will run in the database close to the data in a way that can be easily consumed through the Node Client API alongside all of the builtin capabilities.
> **Enable modern end-to-end JavaScript development**
JavaScript is the most widely deployed language today. Its prominence in browser development has meant that anyone doing any serious development for the web has been exposed to it. This has trickled down to the back-end as well, as evidenced by the rapid growth of Node.js for building middleware. JavaScript in the middle tier means fewer context switches for developers and the ability to share code and data models with the front end with fewer impedance mismatches. This has proven to be a boon for developers. MarkLogic 8 takes this one step further and provides a full JavaScript runtime in the database along with native support for storing and indexing JSON in the database. This means you can easily share data models in JSON and even business logic in JavaScript among the front-end, middle tier, and database.
> **Always open-source on GitHub**
The Node.js Client is developed in the open on GitHub and distributed under the permissive Apache 2.0 license on NPM. This means that you can see how the code works and fork, change, and even redistribute it. The project is led by MarkLogic Engineering and supported by MarkLogic Support, but we encourage anyone to submit issue reports and pull requests.
# Straightforward data integration
> **React faster to change, using data in its natural form with less ETL: JSON, XML, RDF, text, binary**
MarkLogic uses a document data model. Documents can be described as JSON, XML, or even text or binary. Each document can have its own structure. You don’t need to do all of your data modeling up-front, like in a relational database. MarkLogic 8 supports JSON natively as a document format. For Node.js applications, this allows you to easily communicate data back and forth between Node and MarkLogic without brittle translation code. MarkLogic can even index things like dates, geospatial points, and semantic triples that aren’t strongly typed in JSON.
MarkLogic also provides document management and indexing for XML alongside JSON. This unique capability among NoSQL databases allows organizations with existing XML investments to leverage the benefits of Node.js for building applications.
> **Transactional multi-document updates ensure consistency**
MarkLogic’s support for multi-document transactions is also unique among NoSQL databases. A Node.js application reading and writing data to MarkLogic can always be assured that it’s getting a consistent view of the data. This is especially important as you scale your database up or down on the fly. MarkLogic allows you to dynamically add or remove data partitions or nodes in a cluster without changing application code or compromising on consistency. Node is a good choice for scaling the application tier. With MarkLogic you can enjoy this same flexibility at the data tier without having to compromise on durability or consistency in a live system.
> **Async, promises, streams ensure seamlessness with Node**
To the Node developer, the Node Client API makes MarkLogic look like “just another Node service”. We’ve taken care to use patterns and defaults that align with the Node philosophy and best practices. These include never blocking on I/O and leveraging streams and promises to make integrating into other workflows straightforward.
> **Reduce data movement, duplication by moving code to the data and invoking from Node**
The Node.js Client API gives developers access to wide breadth of database functionality in a pure Node interface. It handles all of the plumbing to efficiently communicate with a MarkLogic cluster. From bulk loading, to building queries, to “patch” updates, the API makes it easy to get started and can accommodate a wide variety of applications. For more specialized tasks that require low-level control over performance, the API includes an extension mechanism that allows a developer to easily move logic from front-end code to JavaScript running in the database, close to the data.
> **Reuse JSON data models, JavaScript code across app tiers**
By supporting JavaScript as a runtime and JSON as a data format in both the application in Node.js and the database, developers can reuse models and patterns across tiers. This reduces lossy and oftentimes unnecessary translations between layers and means fewer context switches for developers.
# Deploy in every environment
> **Node.js as standard middleware for connecting JSON services over HTTP with JavaScript**
Node is quickly becoming the go-to for developers looking to stand up small services that they can easily weave together into an application. MarkLogic can help future-proof your development and operations investment as your organization moves from proofs-of-concept to production applications. We also provide a full-featured Java Client and REST services that mean your data won’t be locked into a specific development environment.
> **JavaScript is everywhere: Reuse skills, tools, investments**
One of the reasons that developers are choosing Node is its use of JavaScript. With MarkLogic 8, you’ll be able to more readily tap into those existing skills—or the skills that developers want to move to. Combining something like Angular.js on the front-end, a Node app server, and a JavaScript runtime in the database you get a full-stack JavaScript environment that will reduce training and start-up costs and attract and retain today’s talented developers.
> **Scale up (or down) without modifying application code**
> **Build, test, instrument, debug with standard tools**
—————————————————————————————
# Notes
* Node for “scalability”, but how do you scale your data tier commensurately?
* Presumption of MEAN. Does your dev/ops team look like those that are deploying MEAN into production? Contrast with greenfield. Not an engineered system, a bag of stuff and an acronym. “Don’t be MEAN, MarkLEAN.”
* More Oracle jabs
* Documents are one thing we store, not the only thing
* Succinct wording about extensibility. Not a gap a ninja feature. See also “plugin” in Mongoose: modular could be client-side or server-side.

19. Server-side JavaScript
JavaScript runtime inside MarkLogic using Google’s V8
Run code near the data for unparalleled power, efficiency
Build applications faster from a growing pool of skills, tools
Reduce risk with proven performance and reliability
Decrease brittle ETL and lost fidelity and functionality from JSON data conversions
Pair with Node.js to ease full-stack JavaScript development
Front End
Middle Tier +
Database Layer
Short Description: Live in JavaScript. Run JavaScript near the data for unparalleled power and efficiency with a high performance JavaScript runtime inside MarkLogic using Google’s V8.
Longer Description: Server-side JavaScript allows developers to access the powerful query and data manipulation capabilities of MarkLogic in a language and with tools that they’re already familiar with. Combined with JSON as a native data format, MarkLogic provides an ideal platform for building JSON-based services with JavaScript.
Script for Presentation:
# JavaScript runtime inside MarkLogic using Google’s V8
JavaScript is the most widely deployed programming language in history. As the default scripting environment of web browsers, its rise has mirrored that of the web itself. As a language, it’s flexible enough to be widely applicable—it can be functional or object-oriented and doesn’t require rigid data types or compilation. In healthy competition in the latest battle of the browser wars, Google, Apple, and Mozilla have rapidly increased the performance and reliability of their JavaScript runtimes.
That ubiquity, flexibility, and performance have pushed JavaScript out of the browser and into other parts of the application stack. Node.js, for example, is popular JavaScript middleware for building network services. JSON, a subset of JavaScript, has become a standard for sharing data both outside and inside the firewall.
> Reduce risk with proven performance and reliability
> Run code near the data for unparalleled power, efficiency
MarkLogic provides a JavaScript runtime—the same one that’s in Google’s Chrome browser and Node.js—that’s built into the database. It allows you to write JavaScript that runs as close as possible to the data for maximum performance and control. All of the rich indexes and data management capabilities that have been production hardened in MarkLogic over the last decade are now available in a rich suite of native JavaScript APIs. This means that developers don’t need to learn a new language or a new set of patterns to be productive with MarkLogic. It also means one less context switch in the increasingly web-based development world.
> Build applications faster from a growing pool of skills, tools
> Decrease brittle ETL and lost fidelity and functionality from JSON data conversions
For applications that are producing or consuming JSON. Server-side JavaScript eliminates conversion and ETL that would be necessary in other languages. This is the plumbing work that slows down development and introduces opportunities for bugs.
> Pair with Node.js to ease full-stack JavaScript development
By combining MarkLogic’s Server-Side JavaScript with rich MVC frameworks in the browser, such as Angular, Node.js in the middle tier, MarkLogic provides an ideal foundation for building full-stack JavaScript applications. You might say a “nice” version of the MEAN stack.
Running code in the database is not a new thing. Relational databases have had stored procedures and even application development environments like PL/SQL for decades. MarkLogic has provided a uniquely modern spin on this concept. We provide a high-performance HTTP server in the database itself. This makes publishing web services simple and doesn’t require an extra set of infrastructure to handle the HTTP layer. This is, in fact, how our own Java and Node.js drivers are built.
# Intelligent data layer
With MarkLogic 8, you can build these HTTP services with JavaScript. MarkLogic handles all of the heavy lifting of transactions and parallelization across a database cluster and provides your developers a powerful API in a language that they know and love.
> Maximize performance by bringing code to the data
> Parallel search and aggregates minimize data movement
> Built-in HTTP app server simplifies application architecture
> Reuse JSON data models, JavaScript code across tiers
> Async tasks, batch processing increase flexibility
> Intelligent ES6 iterators, generators improve productivity
We’ve pushed the boundaries on modern JavaScript and have even included some of the powerful new features in the upcoming version 6 of the language itself. These provide additional productivity and performance boosts.
# Better answers from today’s data
> Model and manipulate documents, relationships, and metadata combining JSON, XML, RDF, text, and binary
> Unified JavaScript interface for all indexes, data formats
> * Text search, semantic inference, aggregates, geospatial, alerting
> Real-time consistency when milliseconds count

20. Zap! Zap! Execution Data Model Full-Stack JavaScript JSON JavaScript
Angular
JSON
JavaScript
Zap!
JavaScript object graph
JavaScript
Zap!
Today we'll be working in the middle tier level.
JSON
JavaScript

21. {JavaScript, XQuery} Built-ins
Application Logic
{Node, Java} Client API
Extensions
Application
Network, I/O, Transactions, Builders, Marshaling, Exceptions …
HTTP Data Services
The Node and Java Client APIs are wrappers around the REST Client API, allowing developers to use idioms common in those languages. With the ability to add extensions, the full power of MarkLogic is available.
Let's focus for a minute on how you'd integrate those data services into a three tier application. We'll start with Java, where most of our customers start. However, these concepts could just as easily apply to .NET or PHP or Ruby on Rails.
We provide a Java client API. It's a JAR that you can download or get from Maven today. This API allows Java middleware to communicate with a MarkLogic database in a fluent Java interface. The Java client doesn't require you to learn another query language or unsafely evaluate strings of code, opening you up to injection errors, if you're not careful. From the API, you can manage documents, build sophisticated queries and aggregates and many other things without having to leave the developer's familiar Java environment. This is a really easy way to get going in a Java environment if you know nothing about MarkLogic.
Under the covers the Java API communicates over HTTP to a rich set of REST-style data services that come built into MarkLogic. These out-of-the-box services provide the fundamentals of document management, search, aggregates, and alerting and sufficient for many applications. We've also done a lot of work to make sure that each of these data services encapsulates all of the best practices for performance and scalability.
For the cases where our built-in data services don't provide the right granularity or last-mile performance that your application needs, we provide a comprehensive set of extensibility hooks. Using one of MarkLogic's server-side languages—XQuery today and JavaScript in the upcoming release—a developer can write code that runs as close to the bare metal as possible within the database.
You could imagine a scenario where you need to do a series of updates to many documents, where each step depends on the results of the previous step. You could do this with the built-in query and update APIs over multiple requests. However, bringing documents over the wire at each step might not meet your performance goals. Instead, you can encapsulate all of that logic into an extension—think: stored procedure—and invoke that encapsulated workflow directly from middleware.
The Java API provides an RMI-like data binding facility that allows extension authors to publish endpoints in Java, abstracting away all of the nitty gritty of the client-server communications and data marshalling/unmarshalling. The extension mechanism also allows you to more highly leverage MarkLogic expertise within you development group, by making their data service extensions accessible to any middleware developer, not just those familiar with server-side scripting. One customer, has a small group of MarkLogic expertise that has built out a library of specific stored procedures for performance-critical aspects fo their application. They bundles up these libraries as extensions, each in their own JARs. They publish these JARs to their internal Maven repository, just like any other Java library, where developers can easily access them and include them in their Java projects.
REST Client API
{JavaScript, XQuery} Built-ins
Extensions
MarkLogic
User code
Framework code

22. After today… MarkLogic University: instructor-led and on-demand videos
developer.marklogic.com: tutorials, blogs
docs.marklogic.com: function docs, Guides
Quick Start

23. MarkLogic World 2016 San Francisco | May 9-12 | Park Central Hotel
Tokyo | June 2 | Tokyo Station Hotel
London | June | The IET
world.marklogic.com

24. May 9-12, Park Central Hotel, San Francisco
MarkLogic World 2016
May 9-12, Park Central Hotel, San Francisco
3 Keynotes | 3 Hands-On Technical Workshops
3 MarkLogic University Courses | 4 Days of Content
5 Tracks | 15+ Customer Speakers | 30+ Interactive Sessions
world.marklogic.com

25. MarkLogic World 2016 May 9-12, Park Central Hotel, San Francisco
MONDAY
MAY 9
TUESDAY
MAY 10
WEDNESDAY
MAY 11
THURSDAY
MAY 12
MarkLogic University: Intro to MarkLogic 2-hour Course*
Hands-on Technical Workshops*
Welcome Reception
Opening Keynote
Breakouts
Sponsor Showcase
Expert 1:1’s
Keynote
Partner Summit
*Advanced sign-up required during registration
world.marklogic.com

26. MarkLogic World 2016 June 13-15, The IET, London world.marklogic.com
MONDAY
JUNE 13
TUESDAY
JUNE 14
WEDNESDAY
JUNE 15
MarkLogic University: Intro to MarkLogic 2-hour Course*
Hands-on Technical Workshops*
Opening Keynote
Breakouts
Partner Summit
Sponsor Showcase
Expert 1:1’s
Sponsor Reception
Keynotes
*Advanced sign-up required during registration
world.marklogic.com

27. June 2, Tokyo Station Hotel
MarkLogic World 2016
June 2, Tokyo Station Hotel
Come learn the whys and hows of MarkLogic Enterprise NoSQL from data architecture, implementation, advanced development, and best practices.
Hear from world-class customers and take away new ideas on how you can use MarkLogic to power your organization’s mission-critical applications.
world.marklogic.com

28. APPENDIX

29. MarkLogic / Enterprise NoSQL Database Platform
POWERFUL
AGILE
TRUSTED
Better answers from today’s data
MarkLogic is built to find answers in documents, relationships, and metadata
Adaptive to every environment
MarkLogic runs well everywhere, while preserving the option to change hardware, data, and scale later
Simpler data integration
MarkLogic accelerates and simplifies data sharing across silos, cutting down on ETL and making agile development possible
Hardened, proven platform
MarkLogic has a proven track record of performance under all enterprise conditions
Uncompromised data resiliency
MarkLogic will keep your data safe and whole—no matter what happens in your application or at your data center
The intelligent data layer
An intelligent data layer powers intelligent applications—and makes them faster and more flexible than any alternative
Powerful: Deliver more value, build better apps
Agile: Prepare for and respond to change
Trusted: Enterprise-ready for mission-critical uses
1. Better answers from today’s data
MarkLogic is built to find answers in documents, relationships, and metadata.
Store JSON, XML, RDF, and geospatial data natively
Compose complex queries across many data types - and use flexible, best-in-class indexes to make them fast
Leverage integrated, enterprise-grade full-text search
Utilize all aspects of complex, multi-structured data - from content to metadata to hierarchical data
2. The intelligent data layer
MarkLogic powers intelligent applications - and makes them faster and more flexible than any alternative.
Discover new insights - between time and knowledge with bitemporality, or between existing relationships with Semantic inference
Maximize performance by bringing the code to the data
Make smarter decisions by analyzing real-time data with alerts, matchmaker queries, and more
3. Adaptive to every environment
MarkLogic runs well everywhere from commodity servers to user laptops to the cloud, while preserving the option to change hardware, data, and scale in the future.
Perform on all major platforms and storage options in the enterprise
Blend on-premise, virtualized, and AWS nodes in a single cluster, and scale out without downtime - or back in
Support Hadoop and SQL natively, and connect easily
4. Simpler data integration
MarkLogic accelerates and simplifies data sharing across silos, cutting down on ETL and making agile development possible.
Empower agile development with a data layer that requires minimal data transformation and no static, unchangeable schema
Index and query diverse sets of data as-is, without the need for a master data model or lengthy reconciliation
Avoid the bottlenecks involved in legacy ETL processes
5. Hardened, proven platform
MarkLogic has a proven track record of performance under all enterprise conditions.
Protect your data with the externally certified security features of a traditional relational database, trusted by Fortune 100 companies and large government agencies
Reduce operational overhead with strong monitoring and configuration tools
Rely on support from the 24/7, all-engineer support staff
6. Uncompromised data resiliency
MarkLogic will keep your data safe and whole – no matter what happens in your application or at your data center.
Create applications with a fully ACID transactional, schema-agnostic database
Trust in best-in-class backup and replication features
Ensure 100% uptime during migrations, infrastructure upgrades, and changes in topology

30. // POWERFUL / Deliver more value, build better apps
Store and manage data natively as JSON documents, speeding up development and reducing data transformation with a simplified architecture for end-to-end JavaScript development.
Built-in, lightning fast search and query capabilities across hundreds of billions of documents. And, full-featured UX with type-ahead suggestions, facets, snippeting, relevance ranking, and language support.
Work with new data that didn’t exist before. For example, if John lives in London, and London is in England, then MarkLogic can infer that “John lives in England” and then add that new fact to your semantic search.
Native JSON
Store
Full-text Search
Semantic Inference
Rely on over 30 sophisticated, composable indexes including a universal index, range index, geospatial index, and triple index—all designed so that developers can ask harder questions and get faster responses.
Store and manage data in different tiers based on cost and performance trade-offs, and easily migrate between tiers without any ETL, additional software, or expensive infrastructure changes.
Native XML Store
Store and manage data natively as XML documents, a hierarchical self-describing data type that is ideal for a wide variety of applications.
Tiered Storage
Flexible Indexes
Store RDF triples and query them using SPARQL—providing context to your data and better search with a database that can handle a combination of documents, data, and triples.
Handle historical data along two different timelines, making it possible to rewind the information “as it actually was” in combination with “as it was recorded” at some point in time.
Server-side JavaScript
Live in JavaScript. Run JavaScript near the data for unparalleled power and efficiency with a high performance JavaScript runtime inside MarkLogic using Google’s V8.
Native RDF Triple Store
Bitemporal
“New” badge – indicates features that are new in MarkLogic 8.
Powerful - Deliver more value, build better apps **these are all of MarkLogic’s unique features**
MarkLogic is designed for today’s data, helping you find answers in documents, relationships, and metadata by storing and managing JSON, XML, RDF, Geospatial data, and more. MarkLogic serves as an intelligent data layer, giving you the freedom to do more.
Store geospatial data such as GML, KML, and GeoRSS and do complex queries on the data or in combination with other data types. Also integrate with ESRI ArcGIS and Google Maps for visualization.
Run complex distributed transactions across multiple documents and collections with no performance drop-offs at scale. Production applications run tens of thousands of transactions per second for tens of thousands of users.
Real-time Alerting
Create an unlimited number of real-time alerts by or text using the alerting API and reverse indexes. Whenever a document is loaded that matches a specific query, you’ll know.
Geospatial Support
Fully Transactional

31. // AGILE / Prepare for and respond to change
Scalable and Elastic
Handle petabytes of data without over-provisioning, over-spending, or experiencing downtime, inconsistency, or risk of data loss.
SQL Support
Use a relational SQL data model within MarkLogic, connecting to SQL-based tools using the ODBC driver, or execute SQL commands against relational databases using the MLSAM open-source XQuery library.
MLCP makes it easy to quickly import or export documents and metadata from MarkLogic, or to copy from one database to another using a command-line tool.
MarkLogic Content Pump
Use MarkLogic’s cloud templates to get up and running quickly on AWS or other cloud environments, starting with a three node cluster or a large cluster with over a hundred nodes.
Cloud Ready (AWS)
Run MarkLogic on Windows, Linux, Solaris, OS X. MarkLogic runs easily and is easy to setup in your environment, whether in the cloud, virtualized, or on premises.
XA Transactions
Run distribute transactions across a cluster using the XA (eXtended Architecture) standard, which ensures ACID properties for global transaction processing.
Multi-OS Support
Only use schema when you need it. Ingest all your data as-is, whether structured or unstructured, using the NoSQL document model rather than being forced to use a predefined schema.
Don’t plan your queries in advance of ingesting your data. MarkLogic is designed for search and discovery so that you can run any query at any-time and get real-time results.
Hadoop and HDFS
Make Hadoop better by connecting it to MarkLogic and using it as part of an infrastructure to handle both operational and analytic workloads.
Schema Agnostic
Ad-hoc Queries
“New” badge – indicates features that are new in MarkLogic 8.
Agile - Prepare for and respond to change **these are all of the features that focus on ease-of-use and flexibility**
Enjoy the flexibility of NoSQL to integrate data, and deploy in any environment—whether using Amazon Web Services, virtual machines, or on-premise hardware. With the agility and adaptability of MarkLogic, you can build applications fast.
Configure and administer MarkLogic with a single REST-based API. This provides more programmatic control than ever before—giving DBAs the power and flexibility necessary to run a modern data center.
Get going fast on MarkLogic with Samplestack, an end-to-end three tiered sample application designed to show developers how to implement a reference architecture using key MarkLogic concepts and sample code.
Use multiple indexes in concert across multiple data types—giving you the power to search and query all of your data.
REST API
Samplestack
Index Across Data Types

32. // TRUSTED / Enterprise-ready for mission-critical uses
Scales easily to handle hundreds of terabytes using shared-nothing architecture in which data partitions are completely independent of each other and can act independently.
View and manage the configuration settings for MarkLogic databases, forests, application servers, groups or hosts—and easily propagate changes across the entire cluster.
Restore the database quickly with minimal downtime, relying on full and consistent backups, hot configuration changes, and automatic index optimization without shutting down the system.
Performance at scale
Configuration Management
Customizable Backup
Rely on support from the 24/7, all-engineer support staff to ensure you get answers fast, or just want some friendly tips on saving a few milliseconds on performance.
Have confidence that your data is always available, reducing risk and avoiding interruptions with automated local- or shared-disk failover made possible with shared-nothing architecture.
LDAP and Kerberos
Security
Use third party authentication from LDAP or Kerberos, making the most secure NoSQL database easier to manage.
Customizable Failover
24/7
Engineering
Support
Secure your data with government-grade security. MarkLogic has certified, granular security for modern data governance and to handle the increased complexity of today’s cyber threats.
Don’t settle for a BASE-ic database. Use ACID transactions to ensure you don’t run the risk of encountering data corruption, stale reads, and inconsistent data—all of which are unacceptable.
Point-in-time Recovery
Rollback to a specified point in time by replaying journal archives, an additional feature to ensure disaster recovery and easy of management.
Security Certifications
ACID Transactions
“New” badge – indicates features that are new in MarkLogic 8.
Trusted - Enterprise-ready for mission-critical uses **these are all the features that ensure MarkLogic meets enterprise requirements**
MarkLogic is a hardened platform that is trusted to run mission-critical applications. It has higher security certifications than any other NoSQL database, and has uncompromised data resiliency with features that ensure you will never lose data.
Use the Management API for cluster management, process automation, access controls, database cloning, audit trails, and connections to third-party interfaces.
Flexible Replication
Enable customizable information sharing between systems, allowing for the easy and secure distribution of portions of data even across disconnected, intermittent, and latent networks.
Manage data in collections of documents similar to partitions, called forests, that exist independently and enable scalability and elasticity, rebalancing, efficient operations, and easier data governance.
Monitoring and Management
Atomic Forests