From the Blogosphere

Statistics Canada provides aggregated census, social and economic survey information to the public while protecting sensitive details - and offering multi-years' worth of data online using Sybase technology.

Key Benefits

• Reduces administrative costs and a reduction in human error
• Fully synchronized internal and external secure networks without ever sharing a physical connection or compromising their stringent security
• Improves customer service
• Ability to offer more data products
• Maximizes cost recovery

Sybase Technology

• Sybase Replication Server
• Sybase Adaptive Server Enterprise (ASE)
• Sybase IQ
• Sybase PowerDesigner

Industry

• Public sector

Replication on Demand
Like most countries, Canada, with a population of over 31 million, relies heavily on census information for government and business planning. Census data, and the statistical profiles that can be synthesized from it, are the life blood of a country. Budgets, social programs, and electoral boundaries require a foundation of solid data. Statistics Canada is a federally mandated agency of the Canadian Government charged with collecting, processing, maintaining, interpreting, and disseminating the country's census, social, and economic information. Its mission and data integrity are vital.

To gather data, Statistics Canada conducts a nationwide census every five years and has about 350 ongoing surveys on virtually all aspects of Canadian life. Statistics Canada serves government agencies, business enterprises, and private individuals.

Bidirectional Replication - Fully Synchronized, Forever Separated
Statistical information is a lot like financial data; people care a great deal about its accuracy and confidentiality. At a higher level of information aggregation, confidentiality becomes a non-issue. For instance, your bank would be in serious trouble if they publicized the balance of your savings account, but there is no problem with the bank telling the world the total balance it has under management across all of its depositors. The same is true for Canada's statistical information.

To serve the dual needs of rendering valuable statistical information while maintaining the privacy of its individual citizens and businesses, Statistics Canada has a complex network. Private areas are heavily shielded and completely disconnected from the outside world, whereas the outside world has graduated access to public and subscriber areas. Through a proprietary switching process that takes place frequently, Replication Server keeps the entire network - isolated and non-isolated areas - up to date so both internal and external users have access to current information.

A Highly Secure, Service-Oriented Architecture
To achieve this state of synchronized separation, Statistics Canada uses Sybase Replication Server. Replication Server's queuing mechanism stores the information moved between the internal and external networks. The unique switching approach relies upon Replication Server's ability to always resume its work from where it stopped when it reconnects to the network.

Statistics Canada uses Sybase IQ to hold the current detailed microdata information gathered from the census to store archived census data dating back to 1971. Actually, Statistics Canada has census information dating back to 1666, and has a project underway to add salient data back to 1911 to their online archive. Sybase Adaptive Server Enterprise (ASE) holds census metadata, the data used to describe and interpret the detailed private microdata. Separate ASE servers also contain the disseminated result sets and the information used to drive the public Web site.

Dissemination Services
With the introduction of Sybase Replication Server, all users, including external users, can use a custom tabulation tool to define a custom table specification that is stored along with the metadata. The metadata, containing user's custom tabulations, is replicated to the secure network. Once the metadata is moved to the secure network, the table specifications are run against the microdata and the results are replicated back to the external network. Users can then add the results to their own documents as needed.

Dynamic Archival - Keeps Multi-Year Data Online
Statistics Canada has a vast online repository of data dating back to 1971, some of it for public use and some for the Canadian government and subscription users. Using the capacity of ASE, combined with the speed and compression of Sybase IQ, the information is always online. Statistics Canada reflects a growing trend of moving away from tape archival, with its attendant delays and costs, to keeping all of its information online, dynamically available to users. A project is currently underway to add this to their current online archives.

Jerry MacGillivray talks about cost reductions and service revenue increases Statistics Canada has gained from taking this approach, "With Sybase, we definitely see administrative savings because of the reduced effort to update external metadata and batch processing systems. The biggest benefit for us is data availability. If we have a client who needs to look at 1971 through to 2001 information, it's there. All they have to do is use their desktop to call it. There's no delay for the client and that's the biggest selling feature of our online archive."

With Sybase IQ's unique compression for analytic and high-performance computing, Statistics Canada uses Sybase IQ to manage the microdata repository they run queries against. Sybase IQ runs on an Intel-based Dell machine, with a Silicon Graphics Origin 3400 machine running the metadata and batch processing operations. Sybase IQ compressed their data by 70-80% and improved tabulation performance by orders of magnitude. The result? Requests that used to take as long as three hours dropped down to seconds. Jerry MacGillivray says, "For the 1996-'97 processing cycle, it wasn't uncommon to have backlogs at the end of the day that might take a week or more to clear out. Since we moved to Sybase IQ, we've had zero backlogs. Our tabulations are almost instantaneous."

Efficiency Enables Self-Sufficiency
Statistics Canada receives federal government funding to conduct the census and disseminate a a wide range of standard data products. At the same time, more detailed custom and semi-custom data products are available on a cost-recovery basis. Ray Lackey explains, "We have different levels of access. We have access for the general public, but - while still maintaining confidentiality - we also provide access to deeper levels of detail tables to certain partners and subscribers."

Replication Server cuts costs by reducing the amount of effort spent doing their work. They can work more efficiently on both sides of the network - internal and external. These efficiencies allow Statistics Canada to offer more services to their paying customers and increase their revenue stream, without an attendant increase in cost.

Unflinchingly Robust and Absolutely Correct
Replication Server has a well-deserved reputation for being extremely resilient. Customers comment on its ability to suffer repeated interruptions from network and power outages without missing a beat. When it comes back online, Replication Server picks up the thread exactly where it left off.

Replication Server's dependability leads some customers like Statistics Canada to push the envelope beyond its originally intended uses. To maintain security and to replicate information between the two networks, Statistics Canada purposely disconnects from the network periodically - as if tripping over the network cable numerous times every day - and relies on Replication Server's ability to pick up where it left off and correctly resume its replication tasks.

Ray Lackey remembers Sybase's initial reluctance to bless Statistics Canada's novel architectural approach, "I went to Sybase in 1997 when we were first putting together our replication architecture. I explained that our reality required us to break the connection. Sybase Replication Server was the only product I knew of that would be able to handle it, keep true data integrity, and still send database transactions. The engineers said, 'Well, good luck to you and don't tell anyone you are doing this.' We might be the only customers using Replication Server in this way. We did it and it has been working great for us for years."

Plotting the Course of a Nation
National statistical information is one of the purest uses of computational data. The unstoppable nature and absolute accuracy of the replication process delivered by Replication Server gives Statistics Canada the ability to protect private data, yet disseminate aggregate information, while Sybase IQ provides the ability to keep multi-year data online in a compressed form. Statistics Canada provides a solid foundation of accurate data for Canada's elected representatives, government officials, businesses, unions, non-profit organizations and general populace to make informed decisions as they determine the country's future.