Delphi Clinic C++Builder Gate Training & Consultancy Delphi Notes Weblog Dr.Bob's Webshop
Dr.Bob's Delphi Notes Dr.Bob's Delphi Clinics Dr.Bob's Delphi Courseware Manuals
 Dr.Bob Examines... #3
See Also: other Dr.Bob Examines columns or Delphi articles

InternetExpress (with XML)
One of the most hyped new features of Delphi 5 Enterprise is the XML support, which is most prominently present in InternetExpress. In this article, which was first published in Visual Developer Magazine, Jan/Feb 2000, I'll show you how to make use of InternetExpress and where XML is used at its best - as inter-application communication language.

InternetExpress is based on MIDAS technology and WebBroker technology. MIDAS stands for Multi-Tier Distributed Application Services (Suite), and is the Borland middle-ware technology to connect Application Servers to Clients written in Delphi, C++Builder and JBuilder. WebBroker is the technology to bridge CGI, WinCGI, ISAPI/NSAPI within a single (but multiple-action) Web Module that can be used for all the above. Apart from the integration of MIDAS with WebBroker, InternetExpress adds XML - a new and emerging internet standard.
Unless you've actually worked with WebBroker, MIDAS and/or InternetExpress before, I'm sure this overview didn't really help to clarify the way InternetExpress works. The best way to demonstrate this, I guess, is the Delphi way, by simple doing it (by writing as little code as possible). So, let's just build an InternetExpress example application and get it over with...

Where are your colleagues?
From 1999 to 2001, I worked for (helped to start) a company called TAS Advanced Technologies. At TAS-AT a lot of my colleagues were always "on the move", doing consultancy for a lot of clients in a lot of different locations. It was hard for people "back in the office" to know where someone was at a given day, so it was also hard to make an appointment (or actually talk to someone). Consequently, we felt the need for an application (at first only available on the intranet) that would maintain our colleagues' whereabouts. We called this the AT-WHERE application, and to keep it simple, it consists of only three tables: one with the colleague (or employee) information, one with the week numbers (and actual dates) and one with the diary (days of the week and location) information. These three tables were created as COLLEAGUE.DB, WEEKNR.DB and DIARY.DB located in the ATWHERE database alias (for example in Paradox format using Database Desktop).

MIDAS Server
First, we create the MIDAS Application Server. This application will hold the data and provide it to the clients. We start with a regular application, of which we can modify the main form to display the fact that we're dealing with a MIDAS Server here. Usually, this means resizing the mainform, dropping a TLabel on it with a nice font and caption saying something "AT-WHERE MIDAS Server". Set FormStyle to fsStayOnTop and Position to poDesktopCenter so we'll be able to see it right away when the MIDAS Server starts. Now, save the project as VDM_Server.dpr.
The real work starts when we add a Remote Data Module to the project (from the "Multitier" tab of the Object Repository). Give it a CoClass name of ATWHERE, and leave the Instancing (Multiple Instance) and Threading Model (Apartment) set to their default values. This will result in a Remote Data Module - just like a regular Data Module - a non-visual form to create and manipulate our databases, tables, queries business rules and so forth.
Drop a TTable component on the Remote Data Module and rename it to TableColleague. The new Visual Form Designer will now show three red circles in the left pane. One for the (implicit) Alias, and one around the TableColleague component. These red circles mean that we need to set some additional properties; we're not done, yet. In this case, we need to set the DatabaseName of TableColleague to the ATWHERE alias, and the TableName property to COLLEAGUE.DB. Now, all red circles have disappeared.
You may have noticed the "Data Diagram" by now. This is the place where we can visually inspect table (field) definitions, draw relationships, etc. If we click on it, we see that it's still empty, but we can drag the TableColleague from the left pane to the Data Diagram window. The diagram that is shown for the table is still empty. We need to right-click on the "Fields" leaf in the left pane and select "Add all Fields" in order to create persistent field components, which will immediately show up in the TableColleague object of the Data Diagram.

The TableColleague has a number of interesting fields, like Phone numbers (TAS-AT at work and GSM for mobile phones) and the First and Last name of our colleagues.
For a master-detail relation, we need to add two more tables to the Remote Data Module: WEEKNR.DB and DIARY.DB. Drop another TTable component, name it TableDiary, connect to the DIARY.DB of the ATWHERE alias, and add all fields like we did before. Do the same with WEEKNR.DB. At this time, we can create the connecting master-detail relationship between the TableDiary and TableWeekNr tables, and that's one place where the Visual Data Module Designer is especially handy (apart from being nice to look at).
In order to visually define the master-detail relationship between the TableWeekNr and the TableDiary, we must click on the master-detail button (the second button from the top) and then drag the master table to the detail. This will pop-up the Field Link Designer dialog, where we can specify the master and detail fields (WeekNr in both cases).

When we click on OK, the Visual Data Module Designer will show the master-detail relationship, including a comment tag that specifies the fieldname.
Apart from the WeekNr - Diary master-detail relationship, I'd like to present the name of the colleague to the output as well, so I need it as additional (lookup) field in the Diary detail table. This can be done by adding a lookup field to the Diary table. Click on the lookup-button (the third from the top) and drag the Diary table to the Colleague table. This results in the New Lookup Field dialog, which we should complete as follows:

New Lookup Field

After all this work, the database diagram is defined as follows:

Visual Data Module Designer

Now that we've defined the master-detail and lookup relations, it's time to decide which table(s) to export from the MIDAS Application Server. In this case, that would be the WeekNr master table, and - somehow - the Diary detail records for each week.
To export a table from a remote data module, we need a DataSetProvider component from the MIDAS tab. This component will export the data as XML packets from the Application Server to the InternetExpress Client. Normally, we need one DataSetProvider for each Table, Query or StoredProducer that we want to export from the remote data module. However, when we're dealing with master-detail relationships, we should not export the master and the detail separately. Instead, we should rely on the nested table functionality inside MIDAS, which means that if we export the master table (WeekNr in this case), then for each master record, a new dataset field is added which contains all detail records for that particular master records. Like a sub-table, and hence the term nested-table. This can go on for sub-nests, and in theory we can even go 15 levels deep with Delphi (although I've never tried it deeper than 4 levels, which was confusing enough).
To cut a long story short: we only need one DataSetProvider, connected to the TableWeekNr, which will contain the data from the TableDiary as nested table inside.

Now that everything is in place at the MIDAS Server, we need to run it once, so it can register itself on our machine, and the MIDAS Clients will be able to find it and connect to it (using DCOM for example). Running the Server once just shows the mainform as we designed it:

AT-WHERE MIDAS Application Server

We can close the Server down again. If we should decide to move the Server to another directory on our machine, we only need to re-run it to re-register it at the new location. Quite convenient. The only thing is to remember to cleanup the registry if you decide that the Server will no longer be used (as in demo Servers that you write for an article, for example). We'll find out in a moment what we have to be looking for in the registry in those cases...

InternetExpress Client
Now that the MIDAS Server is ready and available to connect to, it's time to start a new application. This time, we'll create a Web Server Application that will function both as a MIDAS Client and an InternetExpress webpage producer. Consequently, this application will require a web server to execute.
Start a new application, and select the Web Server Application from the Object Repository. Inside the New Web Server Application dialog we can select the type of application: CGI, WinCGI or ISAPI/NSAPI. Regardless of the choice we make, the remainder of the WebBroker application will be the same. The only difference is that a CGI (Common Gateway Interface) application is loaded and unloaded for every request (including the connection to the MIDAS Application Server), while the ISAPI (Microsoft Internet Information Server API) DLL only needs to be loaded in memory once (in the web server memory space), and will remain loaded from that moment on. As a consequence, requests to ISAPI DLLs are handled much faster than requests to CGI executables. The downside of ISAPI is maintainability and integrity. Once an ISAPI DLL has been loaded by the web server, you can't replace it by an update until the web server unloads it (which can be done by stopping the web server itself, for example), and this is not easy to do from a remote location. Furthermore, if the ISAPI DLL should crash, it could leave the host application (the web server) in an unstable state - or bring it down altogether.
Personally, I start with both CGI and ISAPI sharing the same web module. During development, I can deploy the CGI application on my web server (and replace it at will), and debug the ISAPI DLL from within the Delphi IDE - as we'll see near the end of this article.
In order to have both a CGI executable and an ISAPI DLL that share the same web module, we must simply create two projects (in a project group), delete the web module from the CGI executable, and add (share) the web module from the ISAPI DLL with the CGI executable. For the web module itself it doesn't really matter whether or not it's compiled as executable or DLL (it doesn't even know or need to know), and for us it's much more convenient to develop!

Combined with the Midas Server, we should now have three projects loaded in our project group: VDM_Server.dpr, VDM_CGI.dpr and VDM_ISAPI.dpr (full source code is available, of course).
Note that the Web Module looks remarkably like a data module. In fact, it is a data module, with an ActionDispatch component embedded within it. The ActionDispatcher makes sure that any "request" for the web server application gets directed to the correct entry point (or "WebItemAction", as they are called).
But before we concern ourselves with actions, let's first make a connection with the MIDAS Application Server and get our hands on the data we want to publish. Since we used a Remote Data Module at the server side, we can use a DCOMConnection (from the MIDAS tab). Just drop one on the Web Module, and drop down the list of RemoteNames. Somewhere among these names, there must be the VDM_Server.ATWHERE server. The first part is the project name of the MIDAS Server we created earlier, the second part of the name is the CoClass name of the Remote Data Module from that project. This implies that a single MIDAS Server can have more than one Remote Data Module. Once we've selected the ServerName, we can toggle the Connected property. Once it's set to True, the AT-WHERE MIDAS Server will be run and shown on your desktop. This proves that we can indeed find and connect to the MIDAS Server. But now we still need to get the data from the server, and turn it into a web page somehow. This is done by the two components from the InternetExpress tab: XMLBroker and MidasPageProducer. We need one of each, so drop them on your Web Broker. The XMLBroker component has a RemoteServer property, which must be set to the DCOMConnection component. Once this connection is in place, we can drop down the list of ProviderNames to select the DataSetProviders from the remote server. In our case, we have only one - DataSetProvider1 - but again this implies that a given remote data module can have more than one DataSetProvider, as we stated earlier. Once the ProviderName has been assigned, we're ready to proceed to the next step by actually designing the resulting webpage using the MidasPageProducer.

Web Page Editor
If we right-click on the MidasPageProducer, the Web Page Editor pops up. This is the place where we can visually design the resulting webpage. We have a Browser view (based on the Internet Explorer control), an HTML view (showing the resulting HTML source) and two panes on top. The left pane shows the components we create on the MidasPageProducer, while the right pane shows the immediate childs of a selected component on the right side.
Although we can create components here, there's no component palette. I don't know why, but the alternative isn't bad: with a right-click of the mouse again, we get the New Component dialog. This dialog shows the new components that we can create at this location (in the hierarchy of the left pane). When we start, for example, we can choose between a DataForm, a QueryForm or a LayoutGroup. Once we select a DataForm, select that one and right-click again, we can choose between a DataGrid, a DataNavigator, a FieldGroup or a LayoutGroup again. The LayoutGroup, by the way, is nothing special, but merely a way to locate other components (like a FieldGroup on the left of a DataGrid, for example). In practice, I've never used a LayoutGroup, although I used other techniques to layout my webpage, which I'll share with you in a moment.

After we created these four components, the Browser page of the Web Page Editor will show four design-time warnings right above two sets of navigator buttons. The DataNavigators have a warning stating that their XMLComponent = nil, while both the FieldGroup and the DataGrid component have a warning stating that their XMLBroker = nil. The first two warnings are easy to fix: just select the first DataNavigator component and assign its XMLComponent to the FieldGroup (so this navigator will operate on the FieldGroup), and then connect the second DataNavigator to the DataGrid. Now, select the FieldGroup, and assign the XMLBroker component (from our WebModule) to its XMLBroker component. This will get rid of yet another warning, and will also result in a list of three fields in the browser window: WeekNr, Datum and a special FieldStatus1 field. This last field contains a one character status of the current record, and indicates that a record is Inserted, Modified or Deleted. Handy to see - for a developer, that is, but I'm not sure if I want my clients to see this field (not do I expect them to care about it much). By default, all fields from the master table will appear, but we can explicitly specify which fields we want to see by right-clicking on the FieldGroup and adding only those fields that are relevant, such as the WeekNr and Datum fields.
To get rid of the last warning, we can assign the XMLBroker property of the DataGrid to the (only) available XMLBroker component again. This will indeed remove the final warning, but the result is not as intended: the DataGrid will show the captions (= fieldnames) of the master table (WeekNr, Datum, *) and not of the detail table. To take the last step, we must open up the XMLDataSetField property and specify the TableDiary as nested table field. This results in the DataGrid showing the right (detail) captions in the grid. Of course, we don't need every field; I'd like to end-up with the ColleagueName and the days of the week (Monday thru Friday). This time, I'll leave the StatusColumn1 field as well, so we can see when a record has been modified in the browser.

The widths of the columns in the DataGrid depend on the size of the table fields, which are quite wide (80 characters). To narrow the columns, just select all fields and specify 16 as value of the DisplayWidth properties.

 Web Page Editor at Design-Time

Although this is merely a functional webpage, without any special design consideration (colorscheme, graphics, fonts), it'll do just fine for our first deployment and operational test. After all: make it work first, before you make it look good (or work fast, for that matter).

Deployment
In order to deploy the InternetExpress client application, we must make sure we've done a couple of things. First of all, the resulting web server application (in our case VDM_ISAPI.DLL) must end up in a scripting directory on the web server machine, in my case that's d:\www\cgi-bin. The easiest way to make sure the latest version is always in that location, I've set the Output Directory of my project to d:\www\cgi-bin. Due to the fact that IIS caches running ISAPI DLLs, this doesn't always work as the DLL on disk will be locked, but we'll see a workaround for that when we'll get to the debugging section.
Apart from the correct output directory, we must make sure that our web server application actually produces some output in response to a client request. This is done by creating something that's called a WebItemAction. Open up the WebModule and right-click with the mouse inside to bring up the Action Editor. This is the place to define WebBroker Actions (not to confuse with regular TAction components or Action Lists). A WebBroker Action is equivalent to a client request, and we can specify properties as well as an event for these specific actions. The web server application - albeit only one application - can respond to each of the different actions implemented inside. The discriminating factor between these WebItemActions is their PathInfo property: the string immediately following the URL itself (but before any querystring values). In this case, we need to create only one WebActionItem, for which we should set the Default property to True (so regardless of the value of PathInfo in the request, if it doesn't matches, this default action will be executed). We should set the Producer property of the WebActionItem to the MidasPageProducer component from our web module. This also changes the PathInfo property, but since it's the only (and default) web action item, that doesn't matter.
Last but not least, we need to make sure that the generated output can make use of a set of JavaScript files that are needed to parse the generated XML data packets (that will be embedded within the resulting HTML webpage). The JavaScript source files ship with your copy of Delphi 5 Enterprise, and can be found in the Delphi5\Source\WebMidas directory. There are five files, and they total about 60 KBytes of code. The easiest way to make sure that these JavaScript files can be found, is by copying them to a location on the web server, and specifying that location in the IncludePathURL of the MidasPageProducer, which in my case is set to http://192.168.91.201/cgi-bin (since 192.168.91.201 is the IP address of the web server on my laptop).
Once we made sure these last steps have been taken, we can rebuild the VDM_ISAPI project one more time, and execute the resulting ISAPI DLL as http://192.168.91.201/cgi-bin/VDM_ISAPI.DLL? inside Internet Explorer version 4 of higher or Netscape Communicator version 4 of higher.

Debugging ISAPI DLLs
Before we actually run the VDM_ISAPI DLL, I want to share a technique to run and debug an ISAPI DLL without the need for a web server. This even works on a standalone machine, and only requires the presence of Internet Explorer version 4 or higher. With this browser on your machine, there's also an ActiveX control for the "WebBrowser" part of IE, and that part is used by a freeware tool called IntraBob that I wrote some time ago. Like any DLL, we can specify a Host Application to load and run ISAPI DLLs. This time, we need to specify IntraBob.exe as host application, so when we click on the "Run" button, IntraBob is started, and is used to load the ISAPI DLL with help of a webpage that contains a CGI HTML Form to load it. As a consequence, we're free to set breakpoints in our VDM_ISAPI DLL now (although we didn't write any code, this is very useful when trying to debug more complex ISAPI DLLs).

The CGI HTML Form needed to load the VDM_ISAPI.DLL is as follows:

  <HTML>
  <BODY>
  <FORM ACTION="http://192.168.91.201/cgi-bin/vdm_isapi.dll" METHOD="POST">
  <p>
  <INPUT TYPE="SUBMIT" VALUE="Submit">
  </FORM>
  </BODY>
  </HTML>
Once we click on the Submit button, the local version of the VDM_ISAPI.DLL will be loaded, resulting in the following output:

VDM_ISAPI.DLL output in IntraBob

This single page contains it all. HTML with TABLE and INPUT definitions to create the look, XML that contains the master-detail table and field definitions along with the actual data, and JavaScript to be able to parse the XML data and "bind" it to the INPUT control that we see on the webpage. The good news is: it's all there. The bad news is: it's all there!
Regarding the good news: when you browse through the data, you may note that the page doesn't flicker. The HTML stays put, and only the content (value) of the EDIT control changes. This is done by JavaScript code working with the XML data. And that's a pretty big "Oh, Wow!" when you see this for the first time.
Regarding the bad news: when you browse through the data in this webpage, you may notice that all data is accounted for. The entire master-detail recordset has been sent over the wire. Now, consider a really larger customer-order relationship, where the master-detail "join" takes up several megabytes in size. It would take a lot of time - even with my cable modem - to sent that over the wire. Ouch! Fortunately, we can limit the amount of data by modifying the MaxRecords property of the XMLBroker, so we only get the first X records.

You may note that the ColleagueName fields inside the DataGrid look different than the other fields. That's because the ColleagueName is a read-only (lookup) field, and should not be changed. The other fields are edit fields, and Micha (in row 2) just entered his whereabouts on Monday 1999/11/08, which caused the Status field to show the "M" for Modified.
Note that any changes that we make inside the browser will remain there: inside the browser. In order to send our changes back to the MIDAS Application Server, we need to hit the "Apply Updates" button, which will send the updates in another XML packet back to the server, making any updates, and reporting any reconcile errors that should occur (for those of you that want to explore multi-user reconcile errors in more detail: check out the DEMOS\MIDAS\INTERNETEXPRESS\INETXCUSTOM directory for two packages that contain - among others - a ReconcilePageProducer that you can connect to an XMLBroker component to take care of this for you).

Of course, the same result can be obtained using Internet Explorer or Netscape Navigator directly (but in that case you need a real web server). As an added bonus, however, IntraBob does not cache ISAPI DLLs, so we are now free to go back to the Delphi IDE and modify the ISAPI DLL without having to stop and re-start the World Wide Web service on our web server machine.

WebPage Design Issues
The resulting webpage looks a bit gray. A bit like a programmer's webpage, and not at all something that people may want to look at for a long time. For those situations, there are a number of ways to enhance the output. First of all, we can edit the HTMLDoc property of the MidasPageProducer (just leave the #-tags intact). For even more flexibility, we can use the HTMLFile property which points to an external file, overriding the HTMLDoc value. This results in template-based webpage producing, where a third-party can provide (and maintain!) the template for you.
Apart from that, you can manipulate each field in a FieldGroup or each cell in a DataGrid and specify the font, color etc. (all using styles or plain HTML).

Final Words...
InternetExpress is included with Delphi 5 Enterprise and makes use of the MIDAS technology. XML data packets are being sent from the MIDAS Application Server (the DataSetProvider) to the Web Server Application (the XMLBroker). The resulting webpage also contains an XML data packet, and even sends one back to the Application Server (via the Web Server App) when clicking on the "Apply Updates" button.
As such, it means that to deploy an InternetExpress application, a MIDAS license is required.

Latest News: For MIDAS 3, Borland just slashed the suggested list price from $5000 to $299.95 for unlimited usage servers. This is not a typo: less than 300 US-dollars for an unlimited client license per machine (with up to four CPUs), running any number of MIDAS application servers!
Delphi 5 Enterprise comes with a MIDAS 3 developer license, and MIDAS 3 deployment licenses are now available for an extremely competive market price. Thank you Borland!

Conclusions
Delphi 5 InternetExpress uses XML both as inter-application communication (between the MIDAS Server and Web Server Applications) and as data packet inside the resulting webpage. Although some additional support is required for larger dataset, it makes for an excellent technique to quickly and interactively publish data on the web in a truly thin-client way.


This webpage © 2000-2010 by Bob Swart (aka Dr.Bob - www.drbob42.com). All Rights Reserved.