National Association of Realtors

 RETS 1.5 Reference Implementation

Developer's Guide

 

Version 1.5

 

 


Revision History

Date

Version

Description

Author

2002/01/07

1.0

Initial Document

Jeff Brush - Avantia, Inc

2002/10/25

1.5

RETS 1.5 updates

Tom Weber - Avantia, Inc

 

 

 

 

 

 

 

 

 


Table of Contents

1.     Introduction                                                                                                                                                                              5

1.1      Updates for RETS 1.5 Support                                                                                                                                  5

1.2      Other Changes                                                                                                                                                             5

2.     RETS Reference Implementation Architecture                                                                                                                    6

2.1      The Login Transaction Client Side – high level overview                                                                                    6

2.2      The Login Transaction Server Side                                                                                                                          7

3.     RETS Server Implementation                                                                                                                                                  8

3.1      RETSServer                                                                                                                                                                  8

3.2      RETSResponse                                                                                                                                                            8

3.3      RETSServerResponse                                                                                                                                                 8

3.4      RETSResonseFactory                                                                                                                                                 8

3.5      RETSLoginResponse                                                                                                                                                  8

3.6      RETSLogoutResponse                                                                                                                                               9

3.7      RETSGetObjectResponse                                                                                                                                          9

3.8      RETSActionResponse                                                                                                                                                9

3.9      RETSSearchResponse                                                                                                                                                9

3.10        RETSPropertySearchResponse                                                                                                                            9

3.11        RETSAgentSearchResponse                                                                                                                                9

3.12        RETSOfficeSearchResponse                                                                                                                                 9

3.13        RETSGetMetadataResponse                                                                                                                                 9

4.     RETS Client Implementation                                                                                                                                                   9

4.1      RETSConnection                                                                                                                                                       10

4.2      RETSTransaction                                                                                                                                                      10

4.3      RETSLoginTransaction                                                                                                                                            10

4.4      RETSLogoutTransaction                                                                                                                                         10

4.5      RETSGetObjectTransaction                                                                                                                                     10

4.6      RETSActionTransaction                                                                                                                                          10

4.7      RETSGetMetadataTransaction                                                                                                                               10

4.8      RETSSearchTransaction                                                                                                                                          10

4.9      RETSSearchAgentTransaction                                                                                                                               11

4.10        RETSSearchOfficeTransaction                                                                                                                           11

4.11        RETSSearchPropertyTransaction                                                                                                                       11

4.12        RETSLogoutTransaction                                                                                                                                     11

5.     DMQL                                                                                                                                                                                      11

6.     Persistence                                                                                                                                                                              12

7.     Compression                                                                                                                                                                           12

8.     Monitoring and logging                                                                                                                                                        13

8.1      Logging - Apache Log4J                                                                                                                                          13

8.2      RETSMonitor                                                                                                                                                             13


Developer's Guide

1.                  Introduction

The National Association of Realtors has defined a set of standards for the exchange of Real Estate data. The Real Estate Transaction Standard (RETS) consists of a transaction specification and an XML DTD.  See http://www.rets-wg.org  for details of the RETS specification.   The RETS 1.5 Reference Implementation extends the earlier RETS1.0 Reference Implementation and is intended to be used as a testing platform for those who wish to develop servers and clients that conform to the RETS 1.5 specification.  Both a client and server are provided by the reference implementation as well as a limited set of test data.

 

The RETS Reference Implementation is written in Java, the server is a Java servlet and the client consists of API classes and JSP pages.  This software was developed for Apache Tomcat 4.1.2 as the Servlet/JSP engine and MySQL as the database engine.   The RETS Reference Implementation should work with any Servlet engine that supports the 2.2 or 2.3 Servlet Specification and the 1.1 or 1.2 JSP specification.   More information for each of the classes can be found in the API documentation (JAVADOC) provided with the reference implementation.

 

1.1               Updates for RETS 1.5 Support

 

The following changes have been made to the RETS Reference Implementation client and/or server to support the RETS 1.5 Specification :

 

1.2               Other Changes

The following additional changes were made to the Reference Implementation client and/or server to increase performance and stability and to more completely support the standard:


 

2.                  RETS Reference Implementation Architecture

 

Figure 1 show the basic architecture of the RETS Reference Implementation.  The RETS Client is implemented as a web application,  an HTML interface passes user requests to the RETS Client API, the Client API builds and sends RETS Transactions to the RETS Server.   The RETS Server is implemented as a Servlet.  The Data Access Interface provides access to the MySQL datastore.

Figure 1 Architecture Diagram

 

 

2.1               The Login Transaction Client Side – high level overview

 

Figure 2 is a sequence diagram for a login transaction from the RETS client perspective.  “loginAction.jsp” is parsed  when a user selects submit on the login page.  “loginAction.jsp” creates an instance of RETSLoginTransaction and an instance of RETSConnection.  The “execute()” method of the RETSConnection object is called passing the LoginTransaction as an argument.   RETSConnection  sends the request to the RETSServer identified by the “serverURL” variable of the Transaction.  The RETSServer response is parsed by the RETSConnection object and the  “set Response()” method is called on the Transaction object to complete the transaction.  “loginAction.jsp” calls the Transaction’s “getResponseMap()” method and formats the response for display to the browser. 

 

Figure 2 Login Transaction Client side

 

2.2               The Login Transaction Server Side

 

Figure 3 is a sequence diagram for the login transaction on the Server side.  When the RETSServer receives a client request, it creates an instance of a Response object appropriate to the request (transaction) received, in this case a RETSLoginResponse instance is created.  The RETSServer streams the response from the “RETSLoginResponse” back to the client.

 

 

 

Figure 3 Server side of the Transaction

 

 

3.                  RETS Server Implementation

Figure 4 is a class diagram of the RETS Server side classes.  There are actually more classes involved than those shown in this diagram.  Because this reference implementation support both the 1.5 and 1.0 specification, there are two sets of RETSxxxResponse classes, one set in the package “org.realtor.rets.server.v10”, and one set in “org.realtor.rets.server.v15”.    Only one set of classes is shown in the diagram to make it less cluttered and easier to understand.

3.1               RETSServer

Extends HTTPServlet, all client requests are received by RETSServer.  RETSServer deals with the RETS header information and creates an instance of the appropriate RETSResponse object to handle the client request transaction.  User session attributes are maintained by this class. 

3.2               RETSResponse

An abstract class for handling client transaction requests.  Classes extending RETSResponse must implement the buildResponse() method. 

3.3               RETSServerResponse

This class handles requests not recognized or supported by the server.  Sends error response back to the client application.

3.4               RETSResonseFactory

The response factory object builds a RETSResponse object based on the request from the client and level of support claimed by the client.  If a client claims to support version 1.5 of the RETS Specification, the factory create a 1.5 response object.  1.0 response objects are created if the client supports 1.0 of the specification.

3.5               RETSLoginResponse

Builds the response to a login transaction request. The bulk of the work for a login transaction request is currently done in the RETSServer, the server looks up the user in the database and stores the user object as a session attribute.  The RETSLoginResponse simply builds the to the login transaction request.

3.6               RETSLogoutResponse

Builds the response to a logout transaction request.

3.7               RETSGetObjectResponse

Builds the a list of response objects in response to a GetObject transaction request.  RETSServer uses the list response objects to create a multipart response.

 

Figure 4 Server Side Classes

3.8               RETSActionResponse

Builds a reply to a RETS Action request.

3.9               RETSSearchResponse

This class is extended to provide search response support for  “Property”, “Agent” and “Office” searches.

3.10            RETSPropertySearchResponse

Extends RETSSearchResponse to support Property search requests.

3.11            RETSAgentSearchResponse

Extends RETSSearchResponse to support Agent search requests.

3.12            RETSOfficeSearchResponse

Extends RETSSearchResponse to support Office search requests.

3.13            RETSGetMetadataResponse

Builds response to get Metadata transaction request.  The XML file “RETSMetadata.xml” contains the metadata for the RETS Reference Implementation. RETSGetMetatDataResponse searches “RETSMetadata.xml” for the requested metadata object to build the reply.

4.                  RETS Client Implementation

The client is implemented as a series of JSP pages that use the Client API to connect to the server and send transactions.  Figure 5 is a class diagram of the RETS Client API.

4.1               RETSConnection

Used to establish a connection to a RETS Server.  RETSConnection also handles building the required header fields for the RETS Specification (see Section 3) as well as filtering the header information in responses returned from the server.  All transactions are sent to RETS servers through using the “execute()” method of the RETSConnection object.

Figure 5 RETS Client API Classes

 

 

 

4.2               RETSTransaction

Generic RETS Transaction.  Extends “RETSRequestResponse” to deal with transaction responses from the server.  The classes listed below extend RETSTransaction.

4.3               RETSLoginTransaction

Builds the login transaction.  The methods “setUsername()” and “setPassword()” are used to set the required parameters for the login transaction.

4.4               RETSLogoutTransaction

Builds the logout transaction request.

4.5               RETSGetObjectTransaction

Builds the get object transaction request.

4.6               RETSActionTransaction

Builds an Action transaction request to sent a RETS server.

4.7               RETSGetMetadataTransaction

Builds the get metadata transaction request.

4.8               RETSSearchTransaction

Builds the search transaction. This class is extended for each type of search transaction.

4.9               RETSSearchAgentTransaction

Builds search transactions requests for agents. Sets the search type to “Agent”.

4.10            RETSSearchOfficeTransaction

Builds search transaction requests for offices.

4.11            RETSSearchPropertyTransaction

Builds search transaction requests for property.

4.12            RETSLogoutTransaction

Builds the Logout transaction request.

 

5.                  DMQL

Bruce Toback’s DMQLtoSQL was modified to support DMQL and DMQL2  For more information on Bruce Toback’s DMQLtoSQL, see the following URL:http://www.optc.com/~btoback/DMQLtoSQLTest/doc/index.html

Figure 6 shows the classes used to implement DMQL support.   DMQLtoSQLImpl and DMQLtoSQLImpl are modeled after Bruce’s DMQLtoSQLTest class.  DMQLtoSQLImpl and DMQLtoSQLImpl use SQLTableDescrition objects to map RETS objects to the SQL tables used for the reference implementation.  In DMQLtoSQL and DMQL2toSQL, the “translate()” method is used to translate DMQL/DMQL2 query strings into SQL where clauses.

 

The StaticTableDescriptions class stores static table descriptions for Property, Agent and Office searches.  These objects are kept in memory because one or more of these objects are used for every RETS Search transaction,. so it is much more efficient to keep these objects in memory than loading them for each search transaction.

 

Figure 6 DMQL Classes

6.                  Persistence

The reference implementation is delivered with three MySQL tables; AGENTS, OFFICES and PROPERTIES.  Data Access Objects (DAO) are mapped to these tables to provide data for the application.  All database access is provided by the class GenericJDBCDAO, this class basically allows other objects to build a SQL query one piece at a time.  The “FROM” clause in the SQL statement is defined by the class using GenericJDBCDAO, this defines tables and join conditions.  For example the “FROM” statement for RetsListingDAO is an outer join between the property, agent and office tables providing listing information as well as list office and list agent. The “where” clause is built by translating the DMQL or DMQL2 query submitted with the request.

 

Figure 7 shows the classes of the persistence package.  The AttributeList interface provides two methods, getAttrubute(name:String) and getAttributeType(name:String). This just provides a very generic method of accessing objects that are basically a collection of attributes.  SimpleJDBCConnectionPool  provides JDBC connection pooling for the Reference Implementation.

 

RetsListingDAO, RetsOfficeDAO and RETSAgentDAO provide access to the associated content in the database. RetsProperySearch, RetsAgentSearch and RetsOfficeSearch provide convent methods for doing searches using the associated DAO objects.  

 

Figure 7 Persistence classes

7.                  Compression

Compression support for the Reference Implementation is handled by a Servlet Filter.  The reference Implementation supports both GZip and BZip compression   The compression filter processes every response before a reply is sent back  to the RETS Client.  If the client claims that is support GZip or BZip compression, the compression filter will  compress the reply sent back to the client.  The XML below sets up the compression filter for the rets web application, this piece of XML is taken from the web.xml file for the RETS web application.

    <filter>

       <filter-name>Compression Filter</filter-name>

       <filter-class>org.realtor.rets.server.filter.CompressionFilter</filter-class>

       <init-param>

          <param-name>compressionThreshold</param-name>

          <param-value>0</param-value>

       </init-param>

    </filter>

    <filter-mapping>

      <filter-name>Compression Filter</filter-name>

      <servlet-name>RETSServer</servlet-name>

    </filter-mapping>

 

Figure 8 shows the classes involved in supporting compression in the RETS Reference Implementation.  This compression implementation is modeled the intercepting filter design pattern described here :

http://java.sun.com/blueprints/corej2eepatterns/Patterns/InterceptingFilter.html

 

 

Figure 8  Compression Filter

 

8.                  Monitoring and logging

8.1               Logging - Apache Log4J

Apache Log4J is used for logging by all classes in the reference implementation. By default server side classes write to the file “/tmp/retsServer.log” and client side classes write to the file “/tmp/retsClientAPI.log”.  This is configurable by modifying the “log4j.properties” files.  For more information on Log4J see the following web site: http://jakarta.apache.org/log4j

8.2               RETSMonitor

The RETSMonitor class is a servlet that streams the contents of a file to an http client.  The reference implementation uses the RETSMonitor servlet to stream the log4j files to web pages.  By modifying the “log4j.properties” determines the contents of the log files.  If the debug level is set to “info”, only the transaction text is written to the log files. If the debug level is set to “debug” more detailed debug information is written to a log file.