Mining for Data Warehousing Gems

Application of intelligent agent technology for managerial data analysis and mining

Data analysis and mining technologies help bring business intelligence into organizational decision support systems (DSS). While a myriad of data analysis and mining technologies are commercially available today, organizations are seeing a growing gap between powerful storage (data warehouse) systems and the business users' ability to analyze and act effectively on the information they contain. We contend that to narrow this gap effectively, a data analysis and mining environment is needed that can bring together and make available for use many of these technologies, that can support business users with different backgrounds, and with which the users can work comfortably.This paper illustrates the design and construction of such an environment, called the Intelligent Data Miner. IDM is Web-based and it is intended to provide an organization-wide decision support capability for business users. Intelligent agent technology is used as the basis for IDM design. IDM provides several types of data access capabilities to access and analyze the data contained in a data warehouse to obtain the critical information needed by business decision-makers. It supports both predefined and ad hoc data access, data analysis, data presentation, and data mining requests from non-technical users.An operational prototype of IDM, implemented using Java and JATLite (Java Agent Template, Lite from Stanford University), allowed us to examine the feasibility of having the "agents" automatically control and coordinate activities and tasks on the business users' behalf. These agents proved to hide the complexity of data analysis and mining activities, techniques, and methods from the business users, for effective use of the warehouse data.