Business application systems are information systems. They consist of Data and processes (Functions) that act on that data.
Information is the central component to a business and is used to define it. For an effective information system, controls need to be in place that will govern the definition and access (processing) of that information.
This governance involves SME (subject matter experts) defining/approving the data definitions (metadata), as well as the business functionality.
Software (MIS) is used to create the data stores that house the meta-data, and providing access to it as information. Existing systems have the meta-data built from already used components. New systems will have this build be part of the development process.
For RDBMS, this is included within the database catalog tables.
The goal is to produce an accurate, accessible and maintainable Information system in a secure and cost effective manner.
Data Governance is the ways and means of putting controls on the information to achieve this goal.
A batch data processing framework that simplifies, standardizes and optimizes the development and execution process.
This results in a quicker time to market process, with reductions in problems and errors associated with not utilizing a framework.
Java Batch Enterprise(JBE) is a framework that utilizes standard Java classes (POJO), forming the basis of development and execution
A Java Batch application utilizes JBE for easy to use/optimized data access to multiple Sources and Targets, with embedded HTML/Text reporting control.
Some benefits...
• Optimized Batch Processing of enterprise data sources and targets, including ETL and HTML reporting.
• Reduced Cost Savings and Time To Market using industry standard Java and JDBC.
No added software package to buy or depend on for it’s existence.
No Need to train or hire high priced personal to maintain. Use any professional Java developer.
• Reduced Developing and maintenance time and errors. •Reduced Redeployment cost/time of development/executing platforms.
In the future will you need to go from a Windows development platform to UNIX or Linux?
Anywhere the Java JVM runs, and Java tools for development, could be utilized.
• Standardization of development and execution with JBE as the standard.
How is it used?
Development is performed within any Java IDE, including open source Eclipse and IDE’s built upon Eclipse.
These include MyEclipse and the family of Rational Application Developer IDE’s (Rational Developer for Z, WebSphere Application Developer )
Execution can be done via standard batch files *.BAT (Windows), shell scripts (UNIX/LINUX/AIX/USS) or JCL if running on z/OS. A self contained directory structure allows for the running of Java jar files (packages/program), by a simple drag/drop of a data file into a *.bat file, which contains the executing program.
Alternatively, the *.bat file can include the data file name, with no need to drag/drop.
Execution can also be done within the Java IDE. This is an important ability for use in the development process.
For example, to produce an HTML report using an SQL statement as input (or an input File record), only one new class needs to be developed.
This new class would extend an existing java class contained in JBE. The input SQL could be contained in an extended method of the new class, ora file that is referenced in the executing *.bat file or referenced in the Main programs properties file.
Executing the jar/package of this new class would produce the HTML report.
If one needs to be in control of each row returned from the SQL result set (or input file rec), a new method within the new class is created that overrides the behavior of the default report process. The application would then be able to perform any Java operation on each column/field and decide to write out a report record including changed content.
For professional IT projects, all software goes through a company defined process, where documentation is created and approved before any development/changes are started. These often include BDD (business design if software is meant for business line usage) and TDD (tech design) that constitute the Program Requirements. Individual program specifications can then be produced, adding the nonfunctional requirements. Diagram documentation, quite often using UML, is created, so as to define the flow of data and processes. A “picture is worth a thousand words” is very evident here, and diagrams serve as discussion points for interested parties.
Individual programs quite often go through a “walk though” meeting, where the program is presented and looked at by those deemed to be SME’s (subject matter experts) so as to ensure that requirements are being met and adherence to corporate standards.
Approvals are performed by stake holders, such as DBA’s and other system folks, as well as any other downstream software that is dependant/use the results of the new/changed software. This is to done to foresee any possible negative impact that would require a resolution, and to enforce standards
For testing, Use Cases are imperative so as to ensure that all requirements are meet. These are defined by the Business Analysts in consultation with the development team, so as to include the nonfunctional requirements needed for testing. They are also used when future changes are made, serving to ensure that past requirements are still being adhered to. Use cases are not static. They can be added to for new functionality, or changed if past requirements were changed.
While consulting at FISA (NYC financial system), I was giving the responsibility of ensuring that Accounting Interfaces from over 12 NYC agencies, were processed successfully by FISA’s new accounting system (FMS), in format and content.
Input XML files containing an Agency’s accounting interface will feed this process. These files will be validated for proper format and valid content.
Files failing validation will be rejected, while those deemed valid will be inserted to the appropriated Oracle table.
Failed and successfully processed files will have an appropriate message written to the output HTML report.
Java is required. The JDOM class library will be used for parsing of the XML files. JDBC will be used for access to the Oracle database, where the accounting data resides. All reports need to be created in HTML and held on the Web Server for user access in a secure manor.
For an efficient process, database calls will only be made after the transaction passes through the appropriate validation. All SQL Selects are required to have the Primary Key index utilized.
Log4j will be used for logging of errors.
The input XML file is first validated for proper XML syntax using the JDOM API. Badly formed XML files will be rejected with an error report written. No further processing will be performed for this file.
Each data item tag within the properly formed XML file needs to be validated against the associated Oracle table column. The Meta Data repository of Oracle will be used to select all the columns contained within the table. A tag name that has no matching column name will be rejected.
If the table contains a column that has no associated XML tag, then the record will also be rejected.
All files passing validation will have an SQL INSERT statement created, populated with data from the XML tags, and inserted to the Oracle table using JDBC.
- Nonfunctional requirements.
Badly formed XML (or empty file) will be caught as an Exception by JDOM.
All SQL Exceptions will be caught by program, and a database ROLLBACK will be initiated.
The SQLCODE returned from the JDBC INSERT call will be interrogated for a successful completion. Any Exceptions will be written to the error report, ROLLBACK if appropriate, and stop processing.
All caught exceptions need to be reported on, detailing the error with an appropriate message, the method and line # of the exception as well as the process flow to that exception.
If a tag is being rejected, processing will continue with the next tag, so as to have all rejects written to the report file. However, any database Exception will cause the process to ROLLBACK and exit.
Application Processing
Applications are the automated solutions to business processes.
Information sources of paper using file cabinets for storage, are best served as computer data stores (databases, files) and digital displays ranging from Monitors to smart phones. Although paper and physical folders are still used, they are copies of the stored data and not the source.
Enterprise applications require more than a display and database. Complex business practices require a multitude of processing in interactive, transactional and batch modes.
The architectures of information models, processing flows, user and external business interactions need flexibility for a cohesive and maintainable infrastructure. Equally, these architectures require a solid foundation for intensive usage and associated security within their own domains. The soundness of any system is founded in it's least sound component.
Information Modeling
Data is a valued asset of a company. It is not owned by any group or individual but shared by those deemed to satisfy a business need. Security requires that different user roles have different access rights to the data, on a "need to know/process" basis.
Data on it's own is just storage. It becomes useful when it is Information, which is composed of data being presented and manipulated as business functions.
Information that is well defined, utilized and stored in an efficient and secured manor, serves to define how a business functions and its bottom line.
Data Modeling
Defines the relationships among the many data items by grouping them as common attributes. These groups of attributes can be related to other groups by common keyed attributes. These models would then be implemented in an efficient physical model for speed in access and storage. All of this needs to be performed with the security of data as a central focus.
Class Modeling
The means of defining the business processing requirements of its modeled data. The implemented Class model, in effect, becomes the programming (code) of the system. The use of a Class architecture for an applications processing results in minimized development (time to market), maintenance (change of market) and processing exceptions(delays to market).
Interface Modeling
Shared resources among diverse systems is the domain of Interface modeling. The need for a common description of the resource is essential for a workable model. XML is the industry standard language for interfaced data. XSD is the means to modeling these XML interfaces, providing the rules and validation to ensuring a workable architecture.
Information is the result of the implemented Classes interacting with the implemented (physical) data model. All roads to a company's information needs to be through it's Classes, not its physical data model, thus insulating an application from changes to the data model (or system that houses that data model)
Web
Presentation
The web was born with the advent of HTML. Previously, information exchange over telephone lines existed as text based messages, similar in nature to the IM messages of today.
HTML is a tag based language which allows for a developed presentation of information, images, audio and other objects. Data can also be entered, allowing for user interactions with application databases and for social network applications to function. XML is a superset of HTML, which is a specific implementation of it.
Services
Web Services, which are transactional in behavior, allow for the exchange of data in a real time manor over a web based infrastructure, allowing for connection via a URL.
This exchange can occur across the enterprise and as a business to business (B2B) exchange. However, efficiency comes into play and not all data should be exchanged via web services. Batch (FTP) or Message Que exchanges have their place for large scale data exchanges and exchanges not requiring real time information. It should be noted that Message Que hubs (MQ) can behave as the transport layer for web services.
Web 2.0
Social Network sites have changed the web landscape. From being content and data driven, the new web isuser and community driven . Web 2.0 sites are architect to provide an infrastructure that allows for the creation of information by users about themselves and their interests, shared by the community.
Advanced sites allow users to become part of, or even create their own communities of like minded interests, causes and goals.
Collaborative sites, such as Wiki , allow for any user to create or modify existing content for a specific subject matter.
Business Web 2.0
Most Business web sites are nothing more than "store front signs" with pretty images, videos and graphics. They inform of services or products provided and might even display prices, but they live on a self imposed island, offering nothing more than just a "this is what we sell and here is the price".
Websites need to be a reflection of their business. They need to demonstrate subject matter expertise of what they are offering, and allow a user experience of being part of the community the business lives in. This community is made up of not only other users, but employees of the business, suppliers and providers to that business.
Architecture
The defined foundation of any structure or system, where all else is built upon. This can range from the home you live in, to the behavior of an applications system and the structure of it's data stores.
One only needs to know how important a foundation is to one's home, to understand the importance of it in a computing system.
To architect is to design first and build later, with the best architectures often built by those having been the best builders.
