DATA INTEGRATION SUITE RESOURCES

Data Integration Suite Tutorials

Learn valuable skills and pick up tips & tricks about the Data Integration Suite. This section includes free documents with sample code for you to use as examples. Check out the webinars and videos tabs if you don't see what you're looking for here. 

Tutorials & Examples Category List:

 Using the Data Integration Suite

Quick Start - Data Integration Suite:
Utilize the easy to follow examples in this guide to get you started quickly on your tasks with the Data Integration Suite. Samples are provided for the most common data sources and applications – EDI, relational, web services, office documents, and much more.

Data Integration Suite Example Applications:
DataDirect wants you to learn more about how you can get software to work for you. So, we've put together numerous pages that provide a comprehensive set of XML tools and features for working with XML, XQuery, SQL/XML, Web services, XML publishing, and many other XML technologies.

Building XML Workflow Applications:
XML workflow is all about application design. Rather than starting with a database for storing the data, think in terms of the data as it moves around the system; it's a process-oriented rather than data-centric approach. This tutorial explains how to model and develop an XML-workflow application.

Generating Multiple XML Output Files:
Imagine a directory with XML files, each needing to be transformed to a new XML document. This tutorial covers how to query the complete directory and generate the results, all from a single XQuery, with no need for multiple queries, external scripting, or Java code.

Producing HTML and Images:
Need to produce HTML and image output from XQuery? The following tutorial explains how to create an HTML document referencing JPEG files through XQuery. The DataDirect XQuery built-in function ddtek:serialize-to-url is used to save the JPEG images.

 Healthcare Applications

DataDirect HIPAA/ICD Upgrade Toolkit 2009:
To help out with the looming deadlines in Healthcare EDI, DataDirect Technologies now provides the DataDirect HIPAA/ICD Upgrade Toolkit. Read code examples and find out how this toolkit can help you here.

HIPAA 5010 & IDC-10 Compliance Examples:
Working on HIPAA 5010 compliance? Check out thest three healthcare tutorials with cut & paste code to get you jump started.

Converting from HL7 2.x to HL7 3.x:
HL7 versions 2.x and 3.x use different information models. HL7 2.x is EDI-based, 3.x is XML-based, and the concepts are different. This example shows how to go from one to the other, using DataDirect XQuery and DataDirect XML Converters for HL7.

 Converting EDI

Translating X12 documents to XML:
XML Converters are able to handle X12 and many other EDI dialects easily, managing and converting X12 transaction sets, segments, elements, and code lists to XML. XML Converters use their embedded X12 dictionary for helping you create syntactically pure and semantically accurate X12, converting to/from XML or diagnosing problems with incoming and outgoing X12 data.

Converting Custom EDI Message Types to XML:
Vital and valuable legacy data locked in proprietary EDI files? See how to use DataDirect XML Converters™ custom EDI message type tools to convert nonconformant EDI to XML.

EDI Conversion File Polling Service in Java:
Simplify EDI processing in Java — Learn how to write a Java service that listens for EDI files and converts them into XML on-the-fly. Complete with Java example files and detailed instructions.

EDI Conversion File Watcher Service in C#:
Using EDI and .NET? This tutorial shows how to write a Microsoft C# application that listens for EDI files and converts them into XML files. Includes complete C# example files.

Convert Tab Delimited to EDI:
Suppose your company receives tab-delimited messages, and those messages need to be translated to EDI X12 831. Learn how to convert TSV to EDI using DataDirect XML Converters and DataDirect XQuery.

Sample XML Conversions:
How much code does it really take to create a sophisticated XML Converter, like one that converts EDI to XML? Would you believe less than 10 lines of actual code? To see examples in Java, C# and , Visual Basic, click on the above heading.

EDI Translation – How to Transform a directory of EDI messages:
The release of DataDirect XQuery features new integration with with DataDirect XML Converters, and since DataDirect XML Converters support the ability to query all files in a directory, this opens up a new range of use cases and solutions for EDI translation.

 XML & Relational Data

How to Load XML into a Database:
Need to get XML into a Database? Learn how DataDirect XQuery provides an easy and scalable solution for saving or loading XML into any database.

Updating Relational Data:
DataDirect XQuery® supports updates to relational database tables from inside an XQuery by providing three built-in functions. These functions are: ddtek:sql-insert, ddtek:sql-update, ddtek:sql-delete. The DataDirect XQuery® update functionality requires extensions to the XQuery processing model for function declaration, XQuery expressions, and snapshot semantics.

Querying XML Columns:
Increasingly, XML is stored in relational databases as structured XML. Some applications will continue to shred XML into a relational format or store XML as CLOB values. Other applications can now take advantage of relational databases that support a native XML data type to represent an XML document or XML document fragment. This paper shows how you can use DataDirect XQuery® to query XML stored as character data and query XML stored as XML.

MySQL Examples:
Users with MySQL database deployments (Enterprise version) can now take advantage of all the benefits of DataDirect's XQuery engine. The following examples show you how you can use DataDirect XQuery to work with data in a MySQL relational database.

 Web Services & SOA

Creating a Web Service:
Follow along with this tutorial on XQueryWebService that shows how to create a web service using DataDirect XQuery®. View as either as an HTML or as a downloadable PDF. This tutorial has been tested using Tomcat 5.x and 6.x.

An XQuery Servlet for RESTful Data Services:
Find out how to expose XQuery data integration services by invoking them through a Java servlet using a REST interface in this DevX article.

 Working wtih Other Data Sources

Converting Proprietary Flat File Formats to Specific XML Structures:
Proprietary flat file formats are frequently used to exchange or circulate information - Learn how to create create custom Flat File to XML conversions for proprietary formats and how to process the resulting XML data source using XQuery.

Office Documents - Using XQuery with ODF & OOXML:
New office document standards like the OpenDocument Format(ODF) and Office Open XML (OOXML) are making office document integration in business processes a reality. This article discusses using XQuery to query these formats of office documents.

JSON Processing using XQuery:
Choosing between JSON versus XML? It doesn't matter. This tutorial demonstrates how to use DataDirect XML Converters, to transform either JSON or XML with equal facility for use in data processing applications.

Accessing Legacy Data Sources:
Use this online tutorial to see for yourself how to build and deploy an XML-based data integration application that accesses multiple legacy data sources using DataDirect XML Converters™.

 Performance and Optimization

Performance Tips:
DataDirect XQuery® was designed for performance and scalability, which are critical to our customers. This document describes some of the performance optimizations in our XQuery implementation and provides guidelines that will make most applications perform well.

How to Split XML Files:
A common challenge in XML development is splitting a large XML document in several smaller ones. Learn how to deal with such huge XML documents by leveraging DataDirect XQuery to read and split XML.

JDBC Pooling:
Connection pooling allows your application to reuse connections. DataDirect XQuery® supports connection pooling through JDBC. Learn how to configure JDBC connection pool managers in several popular application server environments including Tomcat, JBoss, BEA WebLogic, and IBM WebSphere Application Server.

 Learn XQuery

XQuery Examples:
Download example documents for free and learn how easy it is to integrate, query and publish heterogeneous data sources using the DataDirect Data Integration Suite's XQuery product.

Using XQuery:
W3C XQuery is a query language for XML. The most common use cases for XQuery involve XML publishing to create XML for Web messages, dynamic web sites, or publishing applications. The original data may be found in XML files, or it may be in a data store such as a relational database.

Quick Start – DataDirect XQuery:
This quick start provides basic information for getting started with DataDirect XQuery® immediately after installation. In addition, this quick start describes the DataDirect XQuery® command-line utility that is available for quickly running and testing XQueries through a console window.

DataDirect XQuery® Overview Examples:
The following documents describe how to install, configure and run DataDirect XQuery® examples. Download these examples for free today and learn how easy it is to integrate, query, and publish heterogeneous data sources using XQuery.

Using DataDirect XQuery:
This tutorial shows how to use DataDirect XQuery and XQJ in your Java application to perform tasks that allow you to process queries that access XML and relational data sources, and return XML results, including Configuring connections to relational data sources, Executing queries, querying data in XML sources and DOM trees, joining data from XML and relational data sources, returning results as DOM trees, SAX event streams, and StAX event streams, and using prepared queries (similar to JDBC prepared statements).

Learn XQuery in 10 Minutes:
This XQuery tutorial written by Dr. Michael Kay is for all those people who really want to know what XQuery is, but don't have the time to find out. It covers what XQuery is, and even how to write and run an example XQuery application, all in just 10 minutes!

The XQuery FLWOR Expression:
FLWOR is acronym made up from the five clauses that make up an XQuery FLWOR expression: for, let, where, order by, and return. This tutorial shows how to write an example XQuery FLWOR expression to join data and format results.

XQuery Functions:
XQuery functions, like any programming language functions are useful as a mechanism for hiding complexity, encouraging code re-use, and for advanced computation. This tutorial shows you how to write your own user-defined functions in XQuery.

Resolving fn:collection() Errors:
No standard way exists to access a relational table in XQuery, so each XQuery implementation defines its own way to do this. DataDirect XQuery® uses fn:collection() to access a relational table. This paper provides guidelines to help you troubleshoot and correct common causes of errors that occur when using fn:collection() in an XQuery query.

Use XQuery from a Java environment:
IBM's Developerworks claims "the easiest way to begin with XQJ is to download a free trial from DataDirect." This article is an introduction to XQuery and XQJ, and clarifies its examples throughout by using DataDirect XQuery.

Introduction to the XQuery API for Java (XQJ):
The XQuery language is designed specifically for XML programming and data integration, and programmers are more productive using XQuery for these tasks. However, many enterprise applications are built on the Java platform, and often require functionality not found in XQuery; for instance, many XML programs need to use the Web Services functionality of J2EE.

XQuery: A Guided Tour:
XML (Extensible Markup Language) is an extremely versatile data format that has been used to represent many different kinds of data, including web pages, web messages, books, business and accounting data, XML representations of relational database tables, programming interfaces, objects, financial transactions, chess games, vector graphics, multimedia presentations, credit applications, system logs, and textual variants in ancient Greek manuscripts.

Data Integration Suite White Papers

The following white papers (reports) have been added to our website to explain new developments concerning the Data Integration Suite. These reports include an explanation of the new development, and are here to help educate you about the Suite and its features.

EDI Gets On The Bus
This white paper demonstrates how you can easily integrate the Progress DataDirect Data Integration Suite runtimes on virtually any JMS implementation.

Taming Data Chaos: Simplify Data Integration with DataDirect Data Integration Suite
Alone or in conjunction with other DataDirect solutions, such as Shadow (for mainframe access), the Data Integration Suite can help reverse rising costs via savings in multiple areas. This paper describes the Suite and its capabilities, and illustrates its value proposition with a variety of customer testimonials and use cases across diverse industries.

Helping the Insurance Industry with ACORD Standards
Learn how DataDirect XQuery is being used to help the insurance industry develop and manage applications that must adhere to ACORD (Association for Cooperative Operations Research and Development) standards for the insurance, reinsurance, and related financial services industries.

VSJ Review: Data Integration Suite
The Data Integration Suite is a broad, sophisticated package that lets you create XML applications with very little coding. The pipeline editor and XQuery mapper are excellent designers and easy to use for application creation. The XQuery processor and converters let you use XQuery in your applications in a very simple way, and anyone used to JDBC will have no difficulty with that step.

Building XQuery Based Web Service Aggregation and Reporting Applications
With a vast amount of data being published in XML format by multiple sources, the need has arisen for an easy and efficient means of extracting and manipulating this information. XQuery has emerged as an ideal way to aggregate data from Web services, relational databases, and other applications that employ XML.

DataDirect XQuery Performance: Generating SQL
Implementing XQuery efficiently for relational databases is not trivial, because XQuery and SQL support different operations on quite different data models. At DataDirect, we emphasized performance and scalability in the design of our XQuery engine from the beginning, with a strong focus on relational data. This paper presents some of the techniques we use to generate efficient SQL for relational databases to implement XQuery for these data sources.

Debunking XQuery myths and misunderstandings
You might think XQuery does everything and is well understood, but misconceptions and misunderstandings still exist in the software development community about XQuery. In this article, Frank Cohen details and clarifies many of the myths and misunderstandings that surround XQuery.

Tips and Tricks
Tips and Tricks answers a lot of questions about how you can get the most bang for your buck by using inventive Data Integration Suite coding solutions and, of course, the power and added functionality available from the XQuery processor.

Data Integration Suite Webinars

Below is a list of the most recent Data Integration Suite Webinars. 

Still Transporting Your EDI Data the Same Old Way? Get Up to Speed and Move Data Faster and Better
This webinar demonstrates how DataDirect's Data Integration Suite can help automate mission-critical EDI transactional processing for a variety of commercial industries. Data Integration Suite provides out of the box support for all IATA PADIS, IATA Cargo-IMP, IATA Cargo-FACT, and X12 EDI transactions, including error identification and handling, and custom mapping.

Processing HIPAA Healthcare Transactions
This webinar demonstrates how DataDirect's Data Integration Suite can help automate healthcare transaction processing. The Data Integration Suite provides out of the box support for all HIPAA 4010 & 5010 healthcare transactions, error identification and handling, and custom mapping.

Leveraging the DataDirect Data Integration Suite for Clearinghouse Services
How can the DataDirect Data Integration Suite help deal with a variety of standards, each issuing new versions of their transaction sets once or twice a year? This webinar introduces an approach to dealing with different standards based on XML and data integration through heterogeneous data sources.

Ensuring Customers the Best Data Quality using DataDirect's Data Connectivity Products
DataDirect customer, DataFlux, speaks out about XML
Like most Independent Software Vendors (ISVs), DataFlux is always seeking ways to provide solutions to new and expanding markets. Learn how Scott Gidley, CTO and co-founder of DataFlux uses the DataDirect Data Integration Suite (DDIS) to solve complex business problems.

DataDirect Data Integration Training Webinar
For users most comfortable with an instructor-led learning environment, the DataDirect Data Integration Training Webinar offers a free, convenient, and fast way to get up to speed with all of the components that make up the DataDirect Data Integration Suite. At this webinar, you will learn how the different components of the Data Integration Suite come together in solving a real-world scenario at absolutely no cost.

Data Integration Suite Podcasts

Welcome to Data Integration Podcasts! Here you can download MP3's of the latest podcasts, but be sure to check back regularly because this page is continually updated with new material.

SOA, WOA and Data Services World
In recent years Dr. Carlo Innocenti has been immersed in data integration, particularly integration based on XML technology. At the DataServices World conference in New York City, Dr. Innocenti was an instructor for the Data Access and Data Services Workshop. The workshop included a panel of experts who presented a hands-on session focusing on data access and data services issues, problems and solutions.

SOA – Practical Considerations for Data Integration
This Podcast discusses the steps for solving data integration problems and migrating successfully to an SOA environment. Gain insight into the key trends in business and IT that are making data integration and SOA critical to development. Discover the significant challenges associated with providing effective data access and integration. Learn how SOA is being used to solve data access and security issues and explore the challenges associated with moving to this loosely-coupled architecture.

ZapForum: Business-to-Business Data Integration in a SOA World Presentation
This 1:01:17 podcast features ZapThink analysts Jason Bloomberg and Ron Schmelzer as well as expert guest Jerry King and Patrick Smith. Jerry King leads DataDirect's XML Products group and is responsible for bringing the company's innovative XML technologies to market.

Unleashing XQuery's Power
Download the MP3 of this podcast to get an introduction to XQuery and to find out more about how it is a Web Services enabling technology. Some of the topics covered include: understanding the power and features of XQuery, how does XQuery fit in SOA Architectures, a web services example, how DataDirect XQuery goes above and beyond implementing the XQuery standard, and how will the adoption of XQuery frameworks affect how IT organizations work?

Data Integration Suite Videos

Are you new to XML or Data Integration? Our online video demonstrations provide an easy way to get acquainted with the Data Integration Suite. Choose a topic that interests you – just click to get started. And be sure to check back frequently for new videos on Data Integration and related topics. 

Using the Data Integration Suite 

Data Integration Concepts:
In this video, a DataDirect Technology lead explains the vision of DataDirect XQuery, and discusses topics such as Querying XML and Relational Data, Data Integration, XQJ, XML Converters, and so much more.

Converting EDI 

EDI to XML Conversions:
The Data Integration Suite EDI to XML module (accessible via Stylus Studio) lets you easily convert EDI documents to XML and create custom conversion definitions for non-standard and proprietary EDI documents. The EDI to XML module supports hundreds of EDI dialects and messages types, including ATIS, EDIFACT, HL7, and X12, as well as the SEF (Standard Exchange Format) standard for describing proprietary message types.

• Part 1: Convert a standard X12 EDI document to XML, and customize XML output
• Part 2: Create a custom conversion definition to convert a proprietary EDI document to XML

Using the .NET API for Data Conversion:
Learn how to use DataDirect XML Converters for .NET in a C# project, both as a stand alone application, and in conjunction with XSLT. This tutorial covers using Microsoft Visual Studio .NET integration to build a sample application that converts EDI to XML and XML to EDI and more.

XML & Relational Data 

Updating Relational Data:
This online video tutorial shows how to insert, modify, and delete data in your relational database from your XQueries. Also, see how easy it is to generate Java code that invokes DataDirect XQuery to execute your XQuery application.

Data Integration with Relational Data:
This video shows how easy it is to use Stylus Studio in conjunction with DataDirect XQuery to access relational databases — like Microsoft SQL Server, Oracle, IBM DB2, Sybase, Informix and others — and use relational tables as XML data sources in XQuery.

Web Services & SOA 

Joining Web Services and Relational Data:
This video explains how to use the Data Integration Suite components in order to query and join relational and Web services data. Using DataDirect XQuery in conjunction with Stylus Studio's XQuery Mapper make it easy to use data from disparate sources like relational databases and Web services. Watch this video to find out how.

Learn XQuery 

Unleashing XQuery's Power:
This video is an introduction to XQuery, and how it is a Web Services enabling technology. Some of the topics covered include: understanding the power and features of XQuery, how does XQuery fit in SOA Architectures, a web services example, how DataDirect XQuery goes above and beyond implementing the XQuery standard, and how will the adoption of XQuery frameworks affect how IT organizations work?

Data Intergration Suite Additional Resources

XML Connections Blog
XML Connections is DataDirect's new and improved blog. On it you'll find the latest news and information for XQuery, XQJ, XML Integration, XML Converters, and more.

Developer Forums
This is a collaborative community where visitors share their experience with each other to get the most out of using the Data Integration Suite. If you need further assistance with this product, please visit our Support page, where you'll find an extensive knowledgebase, FAQs and much more.

More Useful Links

Standards Support
The Data Integration Suite supports the following standards: flat-files, EDIFACT, IATA, X12, Edig@s, HIPAA, and EANCOM. To learn more about these standards and to read detailed descriptions of how each standard is supported, click the "Standards" link above.

Product Documentation
View the complete documentation set for DataDirect Data Integration Suite. Also receive tips on viewing and printing the Data Integration Suite product books. All documentation is available in .PDF and HTML format - to view .PDF versions, download Adobe Acrobat Reader here.

Product Documentation for DataDirect XQuery
Product Documentation for DataDirect XML Converters

Frequently Asked Questions
We've created a list of the more common questions that will help you determine that the DataDirect Data Integration Suite is the right solution for you.

News Feeds
DataDirect Data Integration Suite now offers content in popular XML feeds so you can stay up-to-date on news items and posts to our popular mailing lists. Please check details here by clicking the "Feeds" link above.