Web services-based integration (WSBI) at Fairfax County Public Schools
By Ted Davis, Director of Knowledge Asset Management in the Department of Information Technology at Fairfax County Public Schools
Public schools have come a long way since the "little red schoolhouse." Fairfax County, Virginia, Public Schools (FCPS) is the 12th largest school district in the United States, with 234 schools and centers, more than 166,000 students, 20,000 employees, and in excess of 65,000 desktop computers and 800 servers.
Providing a quality education, while achieving an economy of scale for such a large system requires a significant investment in technology. FCPS has enterprise applications covering every major function. Like any large organization there are human resources, payroll, and financial management applications. However, like other school systems, there are also student information, library, transportation, food services, special education, and many instructional applications.
The need for integration among the many applications is great. For instance, the student information application maintains information on students. This information is needed by the transportation application so they can catch the bus. It is needed by the library application so they can check out books. It is need by the food services application so they can pay for meals.
Many school systems are now beginning to integrate applications to reduce redundant data entry. However, at FCPS, enterprise applications have been integrated for many years. The challenge for FCPS is that the integration architecture devolved to a point-to-point "spaghetti" architecture (figure 1). FCPS decided to evolve from the spaghetti architecture to a brokered Web services model, and the effort recently culminated in the deployment of a new summer-school application that integrates with five other applications.
Figure 1: Student Systems "Spaghetti" Architecture
FCPS started this re-architecting effort by forming an Integration Working Group (IWG) comprised of technical leaders representing the enterprise applications. The goals of the IWG were to:
- Establish a scalable architecture for sharing data and services among applications.
- Provide Web access to integrated data and services.
- Seamlessly provide data and services to work processes and eliminate dependency on paper forms.
The first task of the IWG was to gain an understanding of the relationships among the many enterprise applications.
To understand the architecture, the IWG analyzed each application, the interfaces among them, the data shared, and the status of the application.
This analysis highlighted several concerns. At one time a legacy student application, known as CSIS, was acting as a hub among many applications. However, as time moved on and deadlines passed FCPS implemented many point-to-point interfaces. The resulting architecture became very difficult to maintain.
A greater concern was the fact that CSIS was based on an old HP 3000 that would soon be de-supported. Thus, CSIS must be removed without adversely impacting the other applications. Based on the understanding gained by this analysis, the IWG was able to define a solution.
A better way
The IWG determined the best approach would be to incrementally migrate toward a brokered architecture (figure 2). In the brokered architecture each application communicates with the broker. The broker takes on the burden of directing communications and transforming data.
Figure 2: Brokered Architecture
The brokered architecture has the additional benefit of isolating the impact of change. That is, an enhancement to one application need not impact the other applications. This was a severe problem with the spaghetti architecture.
The IWG then conducted a market study to learn what leading vendors might have to offer. They prepared a requirements document and proceeded to the competitive selection process.
FCPS selects a vendor
Adhering to the principle of due diligence, the IWG rigorously evaluated respondents to the request for proposal. The evaluation included an on-site test. The IWG defined the requirements for an integration scenario, created a test environment, supplied the data, and created performance tests. The bidders had five days to implement a solution and execute the performance tests while the IWG observed.
This rigorous approach gave FCPS confidence in selecting the webMethods Integration Platform. webMethods is a leading independent provider of integration software headquartered in Fairfax County, Virginia. FCPS selected webMethods for its ability to solve FCPS integration problems, the ease of use of the development tools, and the close alignment of the webMethods and FCPS strategic direction.
Figure 3: webMethods Integration Platform
Before jumping in and implementing a new, integration architecture, the IWG thought it prudent to conduct an integration exercise known as Integration One. The purpose of the exercise was to establish the hardware environment, establish development practices, and to gain experience with the webMethods technology.
Working with webMethods consultants, the IWG selected the Sun Solaris platform to handle the high-speed communications of an integration platform. The IWG established development, test and production environments (figure 4).
Figure 4: Integration Hardware Environment
The use of Sun Cluster was also critical to this hardware configuration. As a hub for applications, the webMethods platform must be highly available. The hardware and software environment could not become a new single point of failure.
The primary component was the integration server which executes Web services. The enterprise server handles message (document) communication, queuing and routing. An Oracle database was used for logging (figure 5).
Figure 5: Integration Software Environment
With the environment in place the IWG received on-site training on the webMethods platform. They also selected the project to serve as the integration exercise.
The integration exercise needed to be independent of a critical-path delivery to enable a learning experience. However, it also needed to prove that the IWG could integrate information from FCPS enterprise applications. The IWG chose to implement a simple Web application to match teacher information from the student information application with employee information from the human resources application.
Through the course of development, the IWG sought to establish standard practices for:
- Requirements definition
- Integration design
- Data model design
- webMethods implementation
- Configuration management
Configuration management in an integration environment proved to be a challenge. Here the IWG needed to recognize that multiple integrations could progress simultaneously. Furthermore, integrations would advance from development to test to production at different milestones. Thus the environment could not be managed as one would manage a single application. The IWG overcame this challenge by controlling the webMethods environment at the package level for packages associated with a specific integration interface.
Another key outcome was the initiation of a universal data model. In an environment with many applications -- most based on commercial products -- the data modeler's dream of a single enterprise data model often goes unrealized. However, with an integration platform it is possible to define a common representation for enterprise entities, such as student, employee and teacher.
The integration platform enables a universal data model by transforming application-specific representations into the common representation, which is then made available to other applications. This is key to the reuse of services and data.
The IWG developed the Web application in Java Server Pages for an Oracle application server. When a user starts the application, it invokes a webMethods service that retrieves a list of valid schools from the student information application. When a user selects a school, the application invokes a webMethods service that retrieves teacher information from the student information application and another service to retrieve employee information from the human resources system. The information is then matched and presented to the user.
This simple application provided a valuable learning experience. It also provided a valuable utility for preparing data necessary to meet reporting requirements of the No Child Left Behind law.
Now that the fun was over, it was time to do something about the spaghetti architecture. The IWG dubbed this phase of the effort as "Integration X," signifying that this was a project of many projects.
To reduce complexity, the IWG organized the effort into eight integration projects. It then began work on three projects. The primary focus was on the summer-school project. FCPS was in the process of replacing this outdated application (also based on the old HP 3000).
FCPS uses the summer-school application to register students, collect fees, develop schedules, take attendance and assign grades. It was a good focus for integration since it interfaces with the student information, library, transportation, food services and special education applications.
One requirement of the summer-school application was that when a clerk registers a student, the clerk would enter the student name or student ID. At this point the registration screen would be pre-populated with the student's demographic information, which resides on the student information application.
FCPS was implementing the summer-school application in PowerBuilder and Oracle. To meet the integration requirements, the IWG needed to determine how the PowerBuilder application would invoke the services available via webMethods. The IWG wanted to standardize on Web services, however, FCPS was using an older version of PowerBuilder, which did not understand Web services. Other choices include COM, Java, and C++ for the interface development.
Fortunately, the IWG thought of a way to use Web services, and do so in a manner that the PowerBuilder developers easily understood. The approach was for the PowerBuilder developers to invoke Java stored procedures in Oracle, which in turn invoked the Web services.
In this case PowerBuilder invokes a stored procedure which then invokes the "requestStudentCount" Web service. webMethods then retrieves the information from the student information application via the "replyStudentCount" service. webMethods converts all data to the common format and returns the data to the PowerBuilder application (figure 6).
Figure 6: Request and Response Web Service
The integration developers created the Java stored procedure by first creating the service in webMethods. They then used webMethods to generate the Web Services Description Language (WSDL) file. They used Oracle JDeveloper to import the WSDL file and then build the Java stored procedure that invoked the Web service.
Birth of a new architecture
The request-and-reply form of messaging is appropriate for synchronous communications and lends itself very well to Web services (figure 7).
Figure 7: Publish-and-Subscribe Messaging
The IWG also made use of the publish-and-subscribe form of messaging. In this case, triggers update the Student Information Buffer when a student's data is changed. This buffer maintains a current snapshot of student data. It is necessary because the student information application is unavailable for a week that corresponds to the peak summer-school registrations. The webMethods platform detects changes to the buffer and publishes the data. The data is available to any application that subscribes to the corresponding message. Not only does this allow the summer-school application to be informed of data changes, but any application that subscribes, such as special education, could be updated as well.
The combination of request-and-reply and publish-and-subscribe messaging resulted in the new architecture (figure 8). This is exactly the type of brokered architecture the IWG envisioned at the onset of this effort.
Figure 8: Brokered Architecture for Summer School
Will it perform?
With functional and system testing complete, the IWG knew the architecture functioned as designed. However, it remained to be seen whether it would scale to the load that FCPS required. To avoid any surprises in production, the IWG conducted load testing.
The IWG created a load test environment using Mercury Interactive LoadRunner. They designed the environment to simulate registrations on the summer-school application. This would invoke the Java stored procedures and thus execute the Web services on the webMethods platform.
In an attempt to find the breaking point in the architecture, the test increased the load incrementally until the load was over 100 times the anticipated production load.
The results show that the webMethods CPU utilization increased from a 25 percent average utilization to a 55 percent average utilization -- very good given the high load. The results also show that the summer-school server reached its capacity when the script reached the peak load.
Figure 9: Load Test Results
With good test results, the IWG and summer-school teams moved forward with the deployment of the new integration architecture and summer-school application. During the registration period, FCPS enrolled over 26,800 students -- more students than most public school systems enroll during the regular school year.
An unexpected benefit
During the course of registration, the summer-school team realized it did not have an effective means for informing regular school principals which of their students actually registered for summer school. This is critical since there is a very short time between the release of standardized test results and the start of summer school. Principals must ensure that students in need get the necessary remediation from summer school.
The summer-school application was deployed to about 40 summer-school principals, but the team had no plan to deploy and train the other 200 principals. This left the summer-school team considering whether to print and distribute daily registration reports for all schools-until the IWG realized there was a better solution.
The IWG solved this problem by developing a very simple Web application that invoked a service on the webMethods platform. The service selected the students currently enrolled for a specified school. The summer-school team sent the application's URL to all principals via electronic mail. The principals could then log in at any time to see which students were enrolled.
This effort highlighted the potential for Web services. The IWG developed the application in a matter of days. It filled a critical need that was not best satisfied via a major application, and it required no training. Though FCPS operates many applications, there are also many gaps. Web services can fill these gaps very effectively.
Integration is just the start
Implementation of the new integration architecture in support of the summer-school application was a major stride forward, but there is much yet to do. The IWG is still pursuing the other seven integration projects. However, their goal is to go much further.
FCPS uses about 700 varieties of forms, of which it distributes about 9,000,000 per year. Though many of these forms are available online, the processing of these forms constitutes a significant manual effort.
The IWG's long-term goal is to automate the business processes embodied in these forms. Furthermore, they will tie these forms and process into the back-end systems using webMethods Workflow. Achieving this goal requires an effectively integrated application environment. There may be much work ahead, but the prospects are excellent.
Ted Davis is the Director of Knowledge Asset Management in the Department of Information Technology at Fairfax County Public Schools. His office is responsible for the enterprise information systems that run FCPS as well as paper records and forms.
Copyright 2003. Originally published by webMethods, reprinted with permission.
For more information:
- Looking for free research? Browse our comprehensive White Papers section by topic, author or keyword.
- Are you tired of technospeak? The Web Services Advisor column uses plain talk and avoids the hype.
- For insightful opinion and commentary from today's industry leaders, read our Guest Commentary columns.
- Hey Codeheads! Start benefiting from these time-saving XML Developer Tips and .NET Developer Tips.
- Visit our huge Best Web Links for Web Services collection for the freshest editor-selected resources.
- Visit Ask the Experts for answers to your Web services, SOAP, WSDL, XML, .NET, Java and EAI questions.
- Couldn't attend one of our Webcasts? Don't miss out. Visit our archive to watch at your own convenience.
- Choking on the alphabet soup of industry acronyms? Visit our helpful Glossary for the latest lingo.
- Discuss this article, voice your opinion or talk with your peers in the SearchWebServices Discussion Forums.