Text and String Handling in VFP

Last updated: 01 June 2010

Introduction

This article serves to introduce, illustrate, and explore some of the great ( and not so great ) string handling capabilities of Visual FoxPro.

I always seem to be involved with solving many text-data related problems in my VFP projects. On the surface, handling text isnt very sexy and seemingly not very interesting. I think otherwise, and I hope youll agree.

This document is split into three sections: Inbound is about getting text into the VFP environment so you can work with it. Processing is about manipulating the text, and Outbound is about sending text on its way when youre done.

To illustrate text handling in VFP, I am using the complete text of Tolstoys War And Peace, included on the conference CD as WarAndPeace.TXT, which along with thousands of works of literature, are available on the web, including here among others.

This article was originally written using Visual FoxPro version 6, and has since been updated for VFP 7 and VFP 8.

Some facts about VFP strings

Here are a few things you need to know about VFP strings:

In functional terms, there is no difference between a character field and a memo field. All functions that work on characters also work on memos.

The maximum number of characters that VFP can handle in a string is 16, 777, 184.

Inbound

This section is all about getting text into your processing environment.

Inbound text from table fields

To retrieve text from a table field, simply assign it to a memory variable.

CREATE TABLE Ships ( Name C( 40 ), Desc M )
INSERT INTO Ships ( Name, Desc ) VALUES ( "Exxon Valdez", "Billions unpaid!" )
LOCAL lcShip, lcDesc
lcShip = Ship.Name
lcDesc = Ship.Desc

* Note that it doesn't matter if the field is type Character or Memo.

Inbound from text files

There are many ways to retrieve text from files on disk.

FILETOSTR( cFileName ) is used to place the contents of a disk file into a string memory variable. This is among my favorite new functions in VFP 6. Its both useful and fast. For example, the following code executes in one-seventh of a second on my 220Mhz Pentium laptop.

LOCAL t1, t2, x
t1 = SECONDS()
x = FILETOSTR( WarAndPeace.TXT )   && 3.2 mb in size
t2 = SECONDS()
?t2-t1, seconds    && 0.140 ( seconds )
?len( x )   && 3271933

In other words, on a very modest laptop ( by todays standards ) VFP can load the full text from Tolstoys War And Peace in one-seventh of a second.

Low Level File Functions ( LLFF ) are somewhat more cumbersome but offer great control. LLFF are also very fast. The following example reads the entire contents of Tolstoys War And Peace from disk into memory:

LOCAL t1, t2, x, h
t1 = SECONDS()
h = FOPEN( "WarAndPeace.TXT" )   && 3.2 mb in size
x = FREAD( h, 4000000 )
t2 = SECONDS()
?t2-t1, "seconds"  && 0.140 seconds
?LEN( x )   && 3271933
FCLOSE( h )

Given the similar execution times, I think we can conclude that internally, LLFF and FILETOSTR() are implemented similarly. However with the LLFF we also have fine control. For example, FGETS() allows us to read a line at a time. To illustrate, the following code reads the first 15 lines of War And Peace into array wpLines.

LOCAL t1, t2, i, h
LOCAL ARRAY wpLines[15]
CLEAR
t1 = SECONDS()
h = FOPEN( "WarAndPeace.TXT" )   && 3.2 mb in size
FOR i = 1 TO ALEN( wpLines )
  wpLines[i] = FGETS( h )
ENDFOR
t2 = SECONDS()
?t2-t1, "seconds"   && 0.000 seconds

FOR i = 1 TO ALEN( wpLines )
  ? wpLines[i]
ENDFOR
FCLOSE( h )

We can also retrieve a segment from War And Peace. FSEEK() moves the LLFF pointer, and the FREAD() function is used to read a range. Lets read, say, 1000 bytes about half way through the book.

LOCAL t1, t2, x, h
LOCAL ARRAY wpLines[15]
CLEAR
t1 = SECONDS()
h = FOPEN( "WarAndPeace.TXT" )   && 3.2 mb in size
FSEEK( h, 1500000 )  && Move the pointer
x = FREAD( h, 1000 )  && Read 1000 bytes
t2 = SECONDS()
?t2-t1, "seconds"   && 0.000 seconds
SET MEMOWIDTH TO 8000
?x
FCLOSE( h )

Inbound from text files, with pre-processing

Sometimes you need to pre-process text before it is usable. For example, you may have an HTML file from which you need to clean and remove tags. Or maybe you have the problem exhibited by our copy of War and Peace, which has embedded hard-returns at the end of each line. How can we create a streaming document that we can actually format?

Often the answer is to use the APPEND FROM command, which imports from file into a table, and moreover supports a large variety of file formats. The strategy always works something like this: You create a single-field table, and you use APPEND FROM ... TYPE SDF to load it

LOCAL t1, t2
CLEAR
t1 = SECONDS()
CREATE TABLE Temp ( cLine C ( 254 ))
APPEND FROM WarAndPeace.Txt TYPE SDF
t2 = SECONDS()

?t2-t1, "seconds"  && 1.122 seconds

Now youre good to go: Youve got a table of records that you can manipulate and transform to your hearts content using VFPs vast collection of functions.

Processing

This section discusses a wide variety of string manipulation techniques in Visual FoxPro. Lets say weve got some text in our environment, now lets muck with it.

Does a sub-string exist?

There are many ways to determine if a sub-string exists in a string. The $ command returns True or False if a sub-string is contained in a string. This command is fast. Try this:

LOCAL t1, t2, x
x = FILETOSTR( "WarAndPeace.TXT" )   && 3.2 mb in size
t1 = SECONDS()
? "THE END" $ x
t2 = SECONDS()
?[ "THE END" $ x], t2-t1, "seconds"    && 0.180 seconds

The AT()and ATC()functions are also great for determining if a sub-string exists, the former having the advantage of being case insensitive and, moreover, their return values gives you an exact position of the sub-string.

LOCAL t1, t2, x, lnAtPos
x = FILETOSTR( "WarAndPeace.TXT" )   && 3.2 mb in size
t1 = SECONDS()
lnAtPos = ATC( "the end", x )
t2 = SECONDS()
?lnAtPos           && 97837
?[ATC( "the end", x )], t2-t1, "seconds"    && 0.110 seconds
?SUBS( x, lnAtPos, 20 )   &&  the end of it...

The OCCURS() function will also tell you if a sub-string exists, and moreover tell you how many times the sub-string occurs. This code will count the number of occurrences of a variety of sub-strings in War And Peace.

LOCAL t1, t2, x, i
LOCAL ARRAY aStrings[5]
aStrings[1] = "Russia"  && 775 occurences
aStrings[2] = "Anna"    && 293 occurences
aStrings[3] = "Czar"    && 4 occurences
aStrings[4] = "windows" && 23 occurences
aStrings[5] = "Pentium" && 0 occurences
x = FILETOSTR( "WarAndPeace.TXT" )   && 3.2 mb in size
FOR i = 1 TO ALEN( aStrings )
	t1 = SECONDS()
	?OCCURS( aStrings[i], x )
	t2 = SECONDS()
	?[OCCURS( "]+aStrings[i]+[", x )], t2-t1, "seconds"  && 0.401 seconds avg
ENDFOR

Locating sub-strings in strings is something VFP does really well.

Locating sub-strings

One of the basic tasks in almost any string manipulation is locating sub strings within larger strings. Four useful functions for this are AT(), RAT(), ATC(), and RATC(). These locate the ordinal position of sub-strings locating from the left ( AT() ), from the right ( RAT() ), both of which have case-insensitive variants ( ATC(), and RATC() ). All these functions are very fast and scale well with file size. For example, lets go look for THE END in War And Peace.

LOCAL t1, t2, x
x = FILETOSTR( "WarAndPeace.TXT" )   && 3.2 mb in size
t1 = SECONDS()
?AT( "THE END", x )   && 3271915
t2 = SECONDS()
?[AT( "THE END", x )], t2-t1, , "seconds"    && 0.180 seconds

You can also check for the nth occurrence of a sub-string, as illustrated below where we find the 1st, 101st, 201st...701st occurrence of the word Russia in War And Peace.

LOCAL t1, t2, x, i, nOcc
x = FILETOSTR( "WarAndPeace.TXT" )   && 3.2 mb in size
FOR i = 0 TO 7
  nOcc = i*100+1
  t1 = SECONDS()
  ?AT( "Russia", x, nOcc  )
  t2 = SECONDS()
  ?[AT( "Russia", x, ]+Transform( nOcc )+[ )], t2-t1  && 0.180 sec on average
ENDFOR

Two other functions are useful for locating strings: ATLINE() and ATCLINE(). These return the line number of the first occurrence of a string.

Traversing text line-by-line

Iterating through text, one line at a time, is a common task. Heres the way VFP developers have been doing it for years: Using the MEMLINES() and MLINE() functions. Like this:

LOCAL x, y, i, t1, t2
SET MEMOWIDTH TO 8000
x = FILETOSTR( home()+"Genhtml.prg" )  && 767 lines
t1 = SECONDS()
FOR i = 1 TO MEMLINES( x )
	y = MLINE( x, i )
ENDFOR
t2 = SECONDS()
?"Using MLINE()", t2-t1, "seconds"  && 22.151 seconds

Thats pathetic performance. 20+ seconds to iterate through 767 lines! Fortunately, theres a trick to using MLINE(), which is to pass the _MLINE system memory variable as the third parameter. Like this.

LOCAL x, y, i, t1, t2
SET MEMOWIDTH TO 8000
x = FILETOSTR( home()+"Genhtml.prg" )  && 767 lines
t1 = SECONDS()
_mline = 0
FOR i = 1 TO MEMLINES( x )
	y = MLINE( x, 1, _MLINE )
ENDFOR
t2 = SECONDS()
?"Using MLINE() with _MLINE", SECONDS()-z, "seconds"  && 0.451 seconds

Now thats more like it a fifty-fold improvement. A surprising number of VFP developers dont know this idiom with _MLINE even though its been documented in the FoxPro help since version 2 at least.

Starting in VFP 6 all this is obsolete, since ALINES() is a screaming new addition to the language. Lets see how these routines look and perform with ALINES().

LOCAL x, i, y, ti, t2
LOCAL ARRAY laArray[1]
x = FILETOSTR( home()+"Genhtml.prg" )  && 767 lines

t1 = SECONDS()
FOR i = 1 TO ALINES( laArray, x )
 y = laArray[i]
ENDFOR
t2 = SECONDS()

?"Using ALINES() and traverse array:", t2-t1, "seconds"  && 0.020 seconds

Another twenty-fold improvement in speed. I think the lesion is clear: If you are using MLINE() in your applications, and you are using VFP 6, then its time to switch to ALINES(). There are just two major differences: First, ALINES() is limited by VFPs 65, 000 array element limit, and second, successive lines with only CHR( 13 ) carriage returns are considered as one line. For example:

lcSource = "How~~Many~~Lines?~"
lcString = STRTRAN( lcSource, "~", CHR( 13 ))
?ALINES( aParms, lcstring )   && 3

But if you use carriage return + line feed, CHR( 13 )+CHR( 10 ), youll get the results you expect.

lcSource = "How~~Many~~Lines?~"
lcString = STRTRAN( lcSource, "~", CHR( 13 )+CHR( 10 ))
?ALINES( aParms, lcstring )   && 5

This is a bit unnerving if blank lines are important, so beware and use CHR( 13 )+CHR( 10 ) to avoid this problem.

Now, just for fun, lets rip through War And Peace using ALINES().

LOCAL x, i, y, z, k
LOCAL ARRAY laArray[1]
x = FILETOSTR( "WarAndPeace.TXT" )

t1 = SECONDS()
FOR i = 1 TO ALINES( laArray, x )   && 54, 227 array elements
	y = laArray[i]
ENDFOR

t2 = SECONDS()
?"Using ALINES() and traverse", t2-t1, "seconds"  && 3.395 seconds

Excuse me, but wow, considering were creating a 54, 337 element array from a file on disk, then were traversing the entire array assigning each elements contents to a memory variable, and were back in 3.4 seconds.

What about just creating the array of War And Peace:

LOCAL x, t1, t2
LOCAL ARRAY laArray[1]
x = FILETOSTR( "WarAndPeace.TXT" )
t1 = SECONDS()
?ALINES( laArray, x )     && 54, 227 array elements
t2 = SECONDS()
?"Using ALINES() to load War and Peace", t2-t1, "seconds" && 2.203 seconds

So, on my Pentium 233 laptop using VFP 6, we can load War and Peace from disk into a 54, 000-item array in 2.2 seconds. On my newer desktop machine, a Pentium 500, this task is subsecond.

Traversing text word-by-word

You could recursively traverse a string word-by-word by using, among other things, the return value from AT( , x, n )and SUBS( , , ) and, if you are doing that, youre missing a great and little known feature of VFP.

Two new functions are great for word-by-word text processing. The GETWORDCOUNT() and GETWORDNUM() functions, return the number of words and individual words respectively.

Lets see how they perform. Lets first count the words in War And Peace.

LOCAL x, t1, t2
LOCAL ARRAY laArray[1]
x = FILETOSTR( "WarAndPeace.TXT" )    && 3.2 mb in size
t1 = SECONDS()
?GETWORDCOUNT( x )   && 565412
t2 = SECONDS()

?"Using GETWORDCOUNT() on War and Peace", t2-t1, "seconds"  && 0.825 seconds

The GETWORDCOUNT() function is also useful for counting all sorts of tokens since you can pass the word delimiters in the second parameter. How many sentences are there in War And Peace?

LOCAL x, t1, t2
LOCAL ARRAY laArray[1]
x = FILETOSTR( "WarAndPeace.TXT" )    && 3.2 mb in size
t1 = SECONDS()
?GETWORDCOUNT( x, "." )   && ( Note the "." )    26673
t2 = SECONDS()

?"Using GETWORDCOUNT() countING sentences in W&P", t2-t1, "seconds"&& 0.803 seconds

GETWORDNUM() returns a specific word from a string. Whats the 666th word in War And Peace? What about the 500000th?

LOCAL x, t1, t2
x = FILETOSTR( "WarAndPeace.TXT" )    && 3.2 mb in size
t1 = SECONDS()
?GETWORDNUM( x, 666 )  && Anna
t2 = SECONDS()
?"Finding the 666th word in W&P", t2-t1, "seconds"  && 0.381 seconds
t1 = SECONDS()
?GETWORDNUM( x, 500000 )  && his
t2 = SECONDS()

?"Finding the 500000th word in W&P", t2-t1, "seconds"  && 1.001 seconds

Similarly to GETWORDCOUNT(), we can use GETWORDNUM() to return a token from a string by specifying the delimiter. Whats the 2000th sentence in War And Peace?

LOCAL x, t1, t2
x = FILETOSTR( "WarAndPeace.TXT" )    && 3.2 mb in size
t1 = SECONDS()
?GETWORDNUM( x, 2000, "." )
t2 = SECONDS()

?"Finding the 2000th sentence in W&P", t2-t1, "seconds"&& 0.391 seconds

Substituting text

VFP has a number of useful functions for substituting text. STRTRAN(), CHRTRAN(), CHRTRANC(), STUFF(), and STUFFC().

STRTRAN() replaces occurrences of a string with another. For example, lets change all occurrences of Anna to the McBride twins in War And Peace.

LOCAL t1, t2, x
x = FILETOSTR( "WarAndPeace.TXT" )   && 3.2 mb in size
? "Words in W&P:", WORDS( x )   && 565412 words
t1 = SECONDS()
x = STRTRAN( x, "Anna", "the McBride twins" )
t2 = SECONDS()
? t2-t1, "seconds"    && 2.314 seconds

?Occurs( "the McBride twins", x ), "occurences"  && 293 Occurences
? "Words in W&P:", WORDS( x )   && 565412 words

Thats over 125 replacements per second, which is phenomenal. What about removing strings?

LOCAL t1, t2, x
x = FILETOSTR( "WarAndPeace.TXT" )      && 3.2 mb in size
? "Words in W&P:", WORDS( x )      && 565412 words
t1 = SECONDS()
x = STRTRAN( x, "Anna" )
t2 = SECONDS()
? t2-t1, "seconds"    && 2.293 seconds
? "Words in W&P:", WORDS( x )  && 565168 words

So it appears that STRTRAN() both adds and removes strings with equal aplomb. What of CHRTRAN(), which swaps characters? Lets, say, change all s to ch in War and Peace.

LOCAL t1, t2, x
x = FILETOSTR( "WarAndPeace.TXT" )   && 3.2 mb in size
t1 = SECONDS()
x = CHRTRAN( x, "s", "ch" )
t2 = SECONDS()
? t2-t1, "seconds"    && 0.521 seconds

Which isnt bad considering that there are 159, 218 occurrences of character s in War And Peace.

However dont try to use CHRTRAN() when the second parameter is an empty string. The performance of CHRTRAN() in these circumstances is terrible. If you need to suppress sub-strings, use STRTRAN() instead.

String Concatenation

VFP has tremendous concatenation speed if you use it in a particular way. Since many common tasks, like building web pages, involve building documents one element at a time, you should know that string expressions of the form x = x+y are very fast in VFP. Consider this:

LOCAL t1, t2, x
x = FILETOSTR( "WarAndPeace.TXT" )   && 3.2 mb in size
t1 = SECONDS()
x = x+ "<b>Drink Milk!</b>"
t2 = SECONDS()
? t2-t1, "seconds"    && 0.000 seconds

The same type of performance applies if you build strings small chunks at a time, which is a typical scenario in dynamic Web pages whether a template engine or raw output is used. For example:

LOCAL t1, t2, x, y, count
t1 = SECONDS()
x = ""
y = "VFP Strings are fast"
FOR count = 1 to 10000
    x = x + y
ENDFOR

t2 = SECONDS()
? t2-t1, "seconds"    && 0.030 seconds
? len( x )            && 200,000 chars

This full optimization occurs as long as the string is adding something to itself and as long as the string concatenated is stored in a variable. Using class properties is somewhat less efficient. String optimization does not occur if the first expression on the right of the = sign is not the same as the string being concatenated. So:

x = "" + x  + y

is not optimized in this fashion. The above line, placed in the example above, takes 25 seconds! So appending strings to strings is blazingly fast in most common situations.

Outputting text

So you've got text, maybe a lot of it, what are your options for writing it to disk.

Foremostly theres the new STRTOFILE() function which creates a disk file wit the contents of a string. Lets write War And Peace to disk.

LOCAL t1, t2, x
x = FILETOSTR( "WarAndPeace.TXT" )   && 3.2 mb in size
t1 = SECONDS()
STRTOFILE( x, "Junk.txt" )
t2 = SECONDS()
? t2-t1, "seconds"    && 0.480 seconds

Which means that you can dish 3+ Mb to disk in about a half-second.

You can also use Low Level File Functions ( LLFF ) to output text. The FWRITE() function dumps all or part of a string to disk. The FPUTS() function outputs a single line from the string, and moves the pointer

LOCAL t1, t2, x
x = FILETOSTR( "WarAndPeace.TXT" )   && 3.2 mb in size
t1 = SECONDS()
h = FCREATE( "Junk.txt" )
FWRITE( h, x )
FCLOSE( h )

t2 = SECONDS()

? t2-t1, "seconds"    && 0.451 seconds

Here again, the similar performance times between FWRITE() and STRTOFILE() are striking, just as they were when comparing FREAD() and FILETOSTR().

Heres an example of outputting War And Peace line-by-line using FPUTS(). Since were using ALINES(), its not that onerous a task. In fact, its very slick!

LOCAL x, h, i, t1, t2
LOCAL ARRAY laArray[1]

x = FILETOSTR( "WarAndPeace.TXT" ) && 3.2 mb in size
t1 = SECONDS()
h = FCREATE( "Junk.txt" )
FOR i = 1 TO ALINES( laArray, x )
 FPUTS( h, laArray[i] )
ENDFOR
FCLOSE( h )

t2 = SECONDS()
?"Total time:", t2-t1, "seconds"  && 3.595 seconds

Conclusion

So, there you have it, a cafeteria-style tour of VFPs text handling capabilities. I personally think that most of the code snippets Ive shown here have amazing and borderline unbelievable execution speeds. I hope Ive been able to show that VFP really excels at string handling.

The Hooks and Anchors Design Pattern

Last updated: 17 June 2009

Introduction

This document describes an object-oriented design pattern called Hooks and Anchors. Consider this a discussion document.

Hooks and anchors is an object society that I first designed and implemented in 1996 and used ever since. It has served me well in numerous desktop and internet applications.

Here's how this paper goes: first we start by examining the class structure of Hooks and hook subclasses.

Next we'll discuss the Anchor class. Anchors, as with real boat anchors, are a type of Hook.

This is a cool conceptual nuance -- why would an Anchor be a subclass of Hook? The placement of hooks and anchors under a common hook superclass, is explained.

Next we'll look at how particular design patterns are conducive to implementations involving with hooks and anchors.

Background

Most good object-oriented frameworks can be described as skeletal structures designed for extension. Developers evolve their base framework(s), refining and extending them over time.

Similarly, all the root frameworks give you so-called hot spots, known as Hook Methods or, as Gamma and Helm call them, Template Methods. To varying degrees, the framework(s) you use also supply a philosophy about usage and extension. Typically the suggested philosophy is: you hook the framework's hook methods to suit your implementation-specific needs.

In the Hooks and Anchors design pattern, the hook is implemented with a hook object society, which is designed at the outset to fully encapsulate work of the hook method. The hook method, therefore, reduces to a hook operation that delegates program flow to the society of hook objects.

In other words, at the program's critical junctures, instead of adding code to specifically named hook methods, Hook and Anchor pattern implementations use a hook class instead, which hides a society of hooks and anchors, placing code in a polymorphically-named action methods of fine grained hook classes.

In the Hooks and Anchors design presented here, the hook class provides a ::Process() method that is called by clients. The ::Process() method handles pre- and post-processing, calling a protected method named ::Execute() method wherein the hook's real deed is done.

Therefore, A Hooks and Anchors society is the sort of thing you call from a hot spot, usually instead of most or all the code you would normally place there.

The crux of the matter is this: since we're talking about VFP here, the Hooks and Anchors society is, of course, natively engineered to be metadata-driven in both composition and execution.

Class Structure

This section focuses on the morphology of the Hooks and Anchors solution framework. We develop the concept of hook chains and anchors, and how they can be simply combined in the same class hierarchy.

UML Development

Using UML notation, Figure 1 illustrates an atomic concept of object-oriented software: two separate classes with a one-way, 1:1 relationship between them. In this case, SomeClass holds a reference to SomeHook through the SomeClass.oHook member property

Figure 1. a fundamental object micro society.

Figure 2 shows a variation of the relationship in Figure 1, except now assume that two collaborating classes are descendents of a common Parent Class. This assumption may seem like a stretch. If the classes descend from a common parent, then the semantics of the relationship can easily be refactored and generalized into the parent class. If on the other hand the classes do not descend from a common parent, then we can get the same effect with code duplication. For now, lets agree to keep things simple, and descend from a common parent, if only for convenience.

Figure 2. First-pass generalization of the object micro society to a common ancestor.

Into the Parent Class of Figure 2 we can now refactor upwards, moving the details of the relationship into the parent class, which leads to the hierarchy illustrated in Figure 3. Its not obvious upon first glance, but the hierarchy in Figure 3 excels at providing components for object chains, like for example the one illustrated in Figure 4. The key to this is the self-referential relationship link on the Hook Parent Class as illustrated in Figure 3.

Figure 3. At left, all the descendants of the Hook Parent Class inherit the self-referential relationship, and this makes them easily assembled together into chains. The diagram thus reduces abstractly to the one on the right, representing a single class with self-referential semantics.

In my experience great things can happen when you sensibly use self-referential structures like those of Figure 3 to create solutions that execute objects sequentially, as illustrated in the class diagram in Figure 4, and also illustrated in the corresponding stair-shaped sequence diagram illustrated in Figure 5.

Figure 4. A chain of N objects linked together. Normally control passes from instances of Hook1 to Hook2 to Hook3, and so on, normally without any intervention from a controlling object.

Figure 5. The sequence diagram for a chain of objects exhibits a stair-shaped interaction pattern.

Implementation Qualities of Hook Chains

In this section we discuss the types of implementations that naturally lend themselves to solutions by hook chains, then well list some of the Gamma and Helm design patterns that can be created with hook chains, and finally well as list and explain some of the problems Ive encountered when implementing hook chains.

Applications of Hook Chains

According to Riel [2]. the stair shaped interaction of hook chains, as illustrated in Figure 5, is expected to be appropriate in the following situations.

• When operations have a strong connection
  • Consists-of hierarchy: country-state-city
  • Info hierarchy: document-section-topic
  • Temporal: ad-order-invoice-deliver
  • Conceptual: person-customer-key account
  • Operations always in the same order
• Decentralized programming is possible

Abstraction of Hook Chains into Fundamental Design Patterns

When we analyze an object chain of two or more objects, we see that it can be applied in several of the most common structural and behavioral Gamma and Helm Design Patterns, some of which are listed in Table 1.

Table 1. Some Patterns implementable with "chained structures" abstractions.

Some Notable Problems with Hook Chains

Given the usefulness of chained structures for creating a variety of micro- architectures, here is are some of the problems that come with using chains. These problems typically get worse as the chain gets longer and/or more diverse:

• Creating and initializing the chain Hook chains can be short or long, and regardless they need to be assembled, sequenced, and configured. This can be done with some sort of abstract factory, and be aided if the hooks know how to chain themselves together. Then consider the desirability of creating chains from metadata. Therefore the process of creating chains needs abstracting.

• Chain execution control (pre-empting, escaping, re-entering) looking at Figure 5, its hard to imagine invoking an automatic sequential chain of activity without all the control required to properly handle errors and other execution issues like escaping and re-entering the chain.

• Coupling between chain elements Coupling comes in many forms, like control coupling (when execution order is important), content coupling (when one object messes directly with the internals of another), data coupling (when specific data needs to be shared between objects), external coupling (when an object depends on other external objects), stamp coupling (when two components modify or access data in the same object). All these coupling hinder to some degree the integrity, flexibility, and reusability of the chain and its elements.

• Selective iteration of the chain The assumption that every object will fire does not always hold. How then to execute a particular sequence of objects if they are physically chained in another sequence?

• Returning multiple values How do you return multiple values, or aggregate the return values, from a single chain of execution that may be composed of several distinct operations.

• Error and notification handling How to handle errors and notifications in a hook chain?

Extending and Enhancing Hook Chains with Hook Anchors

In the previous section we show that hook chains have both good and not-so-good qualities. Can some of the not-so-good qualities be mitigated?

The complement of the stair-shaped interaction diagram of Figure 5 is the fork-shaped interaction pattern, illustrated in Figure 6. This illustrates a prototypical so-called god-object situation, where behavior is centralized in a single object that manipulates the other objects.

Figure 6. When an object has direct control over other objects, the sequence diagram of the interaction exhibits a fork-shaped interaction pattern.

Here are some of the situations that call for a god-objects fork interaction patterns:

• When operations can change order

• When new operations could be inserted

• When centralized control is necessary and unavoidable

Note as well that holding many concurrent references to external objects pretty much comes with being a god-object.

Lets extend the class hierarchy of Figure 3 and enhance it with a new class for coordination and control purposes. In Figure 7, this new god-object class is the Hook Anchor class, which could possesses, among other things:

• A .aHook member array (or if in VFP 8, a reference to collection) to hold zero or more objects of any subclass of Hook Parent Class. Thus a chain of objects can be built in two different ways: as a simple chain of independent objects, or aggregately referenced by an instance of Hook Anchor which will iteratively invoke them.

• A LOAD() method, called upon INIT(), which takes care of creating and assembling objects, or chains of objects.

• The ability to iterate through a list of objects and, between each, test whether to stop the hook iteration.

Figure 7. Extending the class hierarchy, this time introducing Anchors, which are designed to address the problems of free-form hook chains by providing the ability to concurrently hold many hook objects as well as iterate them in custom and controlled ways.

A notable concept to grasp from Figure 7 is that Anchor classes are hooks. Since hooks can reference hooks, and hooks can be chained, all this applies to Anchors as well.

Abstraction of Hook Anchors into Fundamental Design Patterns

Table 2 below lists of the additional Gamma and Helm design patterns especially the creational ones that the Hook Anchor classes bring, to some degree, to the table.

Table 2. Additional Patterns implementable with Hook Anchor abstractions.

Interface

Here are some of the notable methods of the hook class hierarchy, and of the anchor branch of subclasses.

Upsides

Here are some advantages of Hooks and Anchors

• Runtime configurable.
• Process clones and variants are easily created.
• Modular design, construction, and implementation.

Downsides

Here are some downsides with Hooks and Anchors as presented here.

It takes some getting used to, and its not easy to abstract all the operations of a process into logical independent hooks, and to thereafter anchor them and iterate them properly.

Its more complex than simple subclassing and filling hot-spot methods.

Plugging in a new object involves steps that are vulnerable to configuration mistakes without tool support. Using BROWSE on a VFP configuration table is only good for those who already know what they are doing.

Applications Putting it all together

This section serves as an outline of what will be discussed in the session, and quite probably be included with sample programs on the conference CD, or downloadable from my website at http://stevenblack.com.

• Example: Parsers -- Cleaning up HTML (sample with code)
• Processes -- A Shopping Cart
• Localization INTL

Conclusion

This paper briefly describes the major elements of the Hooks and Anchors solution framework. Approaching problems this in this way isnt for everyone, nor is suitable for all situations. For real hot spots, where your application can be expect evolution stress, it can be just the ticket.

Hooks and Anchors Design Pattern, Example 1

MS Excel "Save As HTML" cleanup.

Given you've got Odious Crap HTML From MS Excel like this (view its source), how to cleanse it so it's generic and clean like, say, this (view source)??

Simple. You set up an object society like the one illustrated below, whose life and execution is controlled by HooksConfig.DBF records, listed in the table below the diagram.

Complete source code here: HooksAndAnchors.ZIP

Given Garbage HTML, here's the hook society that cleans it...

....All created and invoked by this simple VFP code....

*-- This sample shows the cleansing of the
*-- odious HTML you get from an MS Excell "Save as HTML".
*--

*-- Environment
SET PROCEDURE TO HooksAndAnchors additive
SET PROCEDURE TO ParserHooks additive
LOCAL lcHTML, loChain

*-- Processing
lcHTML=FILETOSTR("crapfromexcel.htm")
loChain=CREATEOBJECT("HookAnchor","Excel Paste","Root")
loChain.Process( @lcHTML) && clean this HTML pig!

*-- Show the results
STRTOFILE(lcHTML, "CleanHTMLFromExcel.HTM") && Tada
SHELLEXEC("CleanHTMLFromExcel.HTM")

....which is orchestrated by these records in HooksConfig.DBF.

Bibliography

[1] Gamma, E., Helm, R., Johnson, R, and Vlissides, J. (1994), Design Patterns, Elements of Object Oriented Software, Addison Wesley, Reading, MA, ISBN 0-201-63361-2.

[2] Riel, A (1996), Object Oriented Design Heuristics, Addison Wesley, Reading, MA, ISBN 0-201-63385-X.

[3] Class Composition for Specifying Framework Design, S Demeyer, M Rieger, TD M, E Gelsema (http://scg.unibe.ch/archive/papers/Deme99bClassComposition.pdf)

Hook Operations

Last updated: 22 March 2009

Introduction

In this article, I'll introduce a new design pattern called hook operations, which defines mechanisms to make software flexible and extendible. this is the first design pattern i've ever created from scratch, so I welcome any comments or questions you may have about it.

All software design patterns are born from the need for flexibility. But few patterns explicitly address the issue of how to make software extensible or configurable downstream or at run-time. Furthermore, the more a framework or component is reused, the more clients it serves (by definition) and the greater is the stress on it to adapt. Hook Operations are a way of adapting frameworks or components without stressing them unduly.

Before proceeding, here are some definitions:

Hook: a call within software that's explicitly designated to provide future flexibility.

Hook Operation: code that implements a hook.

Hook Operation Design Pattern: defines interfaces and methodologies for hook operations.

For the most part, the Hook Operation design pattern is applicable in both object and non-object oriented programming. I discuss both, but I only allude to the general implementation mechanisms of procedural hooks, and the code examples presented in this article focus only on object-oriented hooks. I'll cover procedural hooks in a future article.

Name

Hook Operation

Intent

The Hook Operation design pattern defines interfaces and mechanisms by which software is engineered to be flexible and extended by people other than its creators.

Motivation

Toolkits and frameworks are composed of general-purpose classes that are later adapted by others. Toolkits provide for code and component reuse, and frameworks provide for architectural reuse. In both cases, flexibility and extensibility are virtues. How to best provide this adaptability?

In Visual FoxPro, instance programming is one way toolkits and frameworks are sometimes adapted. When you work in the Screen Designer, or in non-class objects in the Class Designer, you are instance programming; that is, you are adding user-code to object instances. This is usually fine and it works, but it means the following disadvantages:

• Source code is required,
• the changes requires recompiling, and
• "programming by exception" (example: "our Atlanta office does things slightly differently") isn't easily accommodated.

Subclassing is another common way toolkits and frameworks are meant to be adapted. Adaptation by subclassing, however, presents similar problems:

• source code is required,
• specialized class knowledge is required to mitigate side-effects,
• the potential for over-enthusiastic subclassing, and
• possibility of creating difficult upgrade, support, and maintenance situations.

Moreover downstream subclassing constrains framework creators in future releases. The framework creators have no control of framework subclassing. This puts framework creators in a difficult position: on one hand they want to perpetually improve their classes, but on the other hand each change may break somebody's code.

Hook operations are usually a viable alternative to instance programming and subclassing.

To illustrate a simple hook, consider a control with the click method in Listing 1. The built-in click method is preceded by the hook operation THIS.PreClickHook(). The PreClickHook() method is a good place for users to place custom processing in subclasses without polluting the built-in Click behavior.

FUNCTION Click
	THIS.PreClickHook()
	WAIT WINDOW "Built-in Click Method Starts Now"
	* <100 lines of code>
ENDFUNC

Listing 1. A simple hook. In this example the user can either instance-program or subclass and place code in the PreClickHook() method. Later we'll see other alternatives to implement hooks.

Subclassing, though, can be problematic. If we could implement hooks without subclassing, we would see the following benefits:

• Source code would not be required.
• Modifications would override or augment the original code, but not change it directly.
• Hooks would be less likely to be affected by future changes to the underlying framework classes.
• Adaptation would occur in places specifically designed for this purpose, but outside the class hierarchy, which is easier to support.

Listing 2 shows how we could modify Listing 1 to engineer hooks without subclassing. Instead of delegating to a local method, the hook calls out to an object reference which can be dynamically configured.

FUNCTION Click
	oSomeObjectReference.ClickHook()
	WAIT WINDOW "Built-in Click Method Starts Now"
	* <100 lines of code>
ENDFUNC

Listing 2. Another simple hook. In this example the hook is implemented by reference to another separate object.

Hooks usually do nothing by default; they are used by exception, if at all. A hook is a separate and distinct method or process, reserved for adaptation, in a way separates the adaptation from built-in behavior. The hook is usually invoked in such a way that the built-in behavior can be enhanced or replaced without affecting the original source and without altering the built-in behavior provided by the class.

The key thing to take from the above is hook operations are ubiquitous calls to external code and, by default and for the moment, that external code simply does nothing.

Later in this article I'll illustrate 2 sorts of hook operations. I'll just describe them for now:

Object Hooks are hooks that are fulfilled by attaching behavior objects to instances. See figure 1.

Figure 1: Object hooks illustrated, here for a single Click() method. In diagram A the instance is not hooked and functions as normal. In diagram B the instance is hooked by a hook object whose non-blank methods extend (or possibly override) the code in the instance.

Procedural Hooks are hooks that are fulfilled procedurally. This fulfillment may be a straight program call, but it could also be metadata-driven. See figure 2.

Figure 2: A metadata-based procedural hook illustrated. Here a call to a Hook Manager triggers a lookup which returns the name of a program to execute. Since hooks are used by exception, normally the lookup would return nothing.

Applicability -- where to put hooks

Hooks are appropriate in places where you want a program to be adaptable. Where might that be? Two conflicting forces determine this. On one hand, hook operations are most valuable at critical junctures in the program. On the other hand, usability suggests that hooks should be placed at predictable junctures.

In object-oriented programs, critical junctures are commonly adjacent to significant state transformations. For example, in an accounting application, the place where transactions are posted is a critical juncture. One would hope for a hook either before and after such important state transitions.

Critical junctures are difficult to identify, especially in frameworks -- criticality, after all, depends on the implementation. An irrelevant juncture in one application may be crucial in another. For completeness, all plausible junctures (all major state transformations) would need to be hooked (and documented). This is onerous.

Predictable junctures, on the other hand, are architecturally predictable places. For example, predictable junctures could be defined as where methods begin and end. Predictable junctures are sometimes (but not always) located close to critical junctures, and their predictability makes them very usable.

How close are predictable junctures to critical ones? This is a function code volume: The briefer the method code, the likelier predictable junctures are proximate to critical junctures.

The tradeoff between critical and predictable junctures is a developer-education issue. If implementers are not able to identify where hook operations are placed, then they wont be able to engineer the adaptations they require. The main benefit of hooking at predictable junctures is the hooks are much easier to identify and document.

I find it easiest to place hook operations at predictable junctures rather than fret about placing hooks at every critical juncture. But don't recommend hooking all predictable junctures. Every method of every object doesn't need a hook, and since the hook calls involve processing, it's best to use some common sense. Some common events I hook include Click(), DblClick(), RightClick(), DragDrop(), GotFocus() and a few others. For obvious reasons, I don't recommend hooking quasi-continuous events such as MouseMove() and Paint().

Structure

A number of hook structures are possible, depending on how the hook is engineered.

Internally fulfilled hooks: When hooked objects handle fulfill their hooks by instance programming or by subclassing, the structure illustrated in Figure 3 is typical. This classic structure is well known and is the basis for the Template Method design pattern as described by Gamma, Helm et al in their Design Patterns book.

Externally fulfilled hooks: If hook operations delegate to external hook implementations, this leads to many possible variants. Two examples: Figure 4 shows a case where the hook objects come from separate hook classes, and Figure 5 shows a case where the hook class is a superclass of the hooked class.

Figure 3: What I call "self-sufficient hooks" have a structure similar to that of the Template Method design pattern. Here you use inheritance or instance programming to modify the default behavior.

Figure 4: Supporting hooks are separate hook objects, here shown as separate classes from those they adorn.

Figure 5: Hooked objects and their hooks can share the same interface, which implies that they can share the same class hierarchy. This is very handy from a maintenance perspective since the interfaces remain automatically in synch.

Figure 5 deserves explanation because it is so counterintuitive and admittedly convoluted. One would think that a hook couldn't possibly be from a Parentclass of what we wish to modify. But it works great! Here's why:

Hooked objects use the public interface of hook objects. Therefore giving both a common interface makes sense because of the polymorphic nature of the linkages. Furthermore if they are of the same class, then keeping the interfaces in synch is a cinch.

If a hooked object is-a hook, then the concrete objects can use each other as hooks. Effectively, this means they can call each other's services as hook operations. For example, all the controls on a form could use the form's RightClick() services, preserving the RightClick() context of the form whilst the cursor is on a control.

Since hooked objects can call hooks, and hook objects are hooks, this means they can be chained together. (I'll explain this later in the Implementation section).

Structure: Hook sequencing

Assuming we are free to sprinkle hook operations throughout our events and methods, how to best place them? To help answer this, here are our placement choices relative to the built-in behaviors we may wish to adapt:

Pre-process hooks are invoked before the usual procedure code. Pre-process hooks are especially handy if they can override (or pre-empt or replace) built-in behavior.

Post-process hooks are invoked after the usual procedure code. Unlike pre-process hooks, post-process hooks cannot control the execution of the procedure which invokes it.

Mid-process hooks are invoked in mid-procedure. I won't discuss this option further, but bear in mind that these can be useful, especially within monolithic procedures.

Participants

There are two participants in the hook operation pattern. Note that the pattern can be extended, so one participant may play both roles.

Hooked Object
Contains a template method which contains one or more hook operations.

Hook Object
Not necessarily distinct from the Hooked Object
Implements the hook

Collaborations

The hooked object hook methods through hook operations.

Hook objects may themselves be hooks.

Consequences

The advantages of Hook Operations include:

• Downstream developers can modify application behavior by adapting hook methods.
• Depending on the architecture, hook objects may be chained together.

Some disadvantages are

• Hook operations are unconventional, and add somewhat to overall application complexity.
• Hook messaging overhead has performance expenses.

Implementation

This section briefly shows some ways hook operations can be implemented in Visual FoxPro

1. A simple pre-process hook

Listing 3 shows a simple call to a pre-process hook. This example uses the common convention of naming the pre-process with the prefix "pre", and similar hook operations are found in a number of VFP frameworks.

FUNCTION Click
  *-- Pre-process hook
  THIS.PreClick()
  WAIT WINDOW "Click"
  * <100 lines of code>
ENDFUNC

Listing 3. A pre-hook followed by standard method code.

2. A simple post-process hook

Listing 4 shows a simple post-processing hook. This example uses the common convention of naming the pre-process with the prefix "post"

FUNCTION Click
  WAIT WINDOW "Click"
  * <100 lines of code>
  *-- Post-process hook
  THIS.PostClick()
ENDFUNC

Listing 4. A post-process Click hook.

3. Combined simple pre and post-process hooks

As you may have guessed, pre and post-hooks can be combined, as shown in listing 5, in an attempt to achieve flexibility on both sides of the standard call. This is not recommended!

FUNCTION Click
  THIS.oHook.PreClick()  && Pre-hook
  WAIT WINDOW "Click"
  * <100 lines of code>
  THIS.oHook.PostClick() && Post-hook
ENDFUNC

Listing 5. Pre and post-hooks surround a standard method. If you intend this method to be subclassed, then avoid this!

However you do it, combining pre and post-hooks, as in listing 5, is not recommended if the method is to be subclassed. If you subclass the click method of listing 5, and use DODEFAULT() or the scope resolution operator (::) to invoke this parent method, then youll fire both the pre and post-hook at that point. Depending on the hooks, this can cause unexpected results because both hooks will fire as part of the scope resolution invocation.

4. A delegated pre-process hook

In a generalization of Listing 3, we could substitute THIS.oHook to reference a specialized hook object, as illustrated in Listing 6.

FUNCTION Click
  *-- Pre-process hook
  THIS.oHook.PreClick()
  WAIT WINDOW "Click"
  * <100 lines of code>
ENDFUNC

Listing 6. A pre-hook method belonging to a hook object. This gives us flexibility to substitute behavior objects, as well as accomplish the same thing as Listing 3 by setting THIS.oHook=THIS.

5. A simple polymorphic pre-hook

Listing 7 improves on Listings 3 and 6 because it uses a clean implementation heuristic: Hooks are called polymorphically. In other words, the Click() method of an object calls the Click() method of its hook.

FUNCTION Click
  *-- Pre-process hook
  THIS.oHook.Click()
  WAIT WINDOW "Click"
  * <100 lines of code>
ENDFUNC

Listing 7. A cleaner version of Listings 3 and 6. Hook methods are called polymorphically.

6. A controlling pre-process hook

Listing 8 shows a flexible variant of listing 7 where the pre-processing hook's return value controls the standard method execution. This is a crude NODEFAULT-like implementation, controlled by the pre-hooks return value.

FUNCTION Click
  IF THIS.oHook.Click() && Pre hook
    WAIT WINDOW "Click"
    * <100 lines of code>
  ENDIF
ENDFUNC

Listing 8. A pre-process hook method (in a separate object) controls the execution of the standard method.

7. Separated pre- and post-process hooks

Listing 9 dramatically improves on Listing 5 because it uses both pre and post hooks, allowing you to completely bracket transformations. Listing 9 uses the following heristic:

1. Events are pre-hooked,
2. events then call a method, and
3. that method is post-hooked.

In this separated "pre" and "post" architecture it is much safer to subclass the event or the method without scope-resolution (::) entanglement.

FUNCTION Click
  *-- Pre-process hook
  IF THIS.oHook.Click()
    THIS.ClickImplementation()
  ENDIF
ENDFUNC
FUNCTION ClickImplementation
  *-- The actual click implementation
  WAIT WINDOW "Click"
  * <100 lines of code>
  *-- Post-process hook
  THIS.oHook.ClickImplementation()
ENDFUNC

Listing 9. A dramatically more flexible version of Listing 5. The pre-hook is called before the event code, the event code calls a method, and a post hook ends the method. Thus we have both pre- and post-hooks and few of the subclassing problems we'd have if both hooks were in the event code.

I personally think that code structured like that of listing 9 is very slick. Note that the event (like click() and dblclick()) fires the "pre" hook predictably near the beginning of its execution, and the method (the thing invoked by event) fires the "post" hook predictably the end of its execution. Thus we have, in effect, a hook on the way in, and another on the way out. Other advantages:

• Code in the user-interface object's events is kept to a minimum, thus making it easier to reuse.
• The behavior of the user-interface object can be totally controlled by the pre-process hook.
• Pre and post hooks can coexist in the same process without any inherent subclassing problems.
• The locations of the all such hooks -- here and elsewhere -- is easily predictable with a simple heuristic: Events fire "pre" hooks, events then fire methods, and the methods fire "post" hooks..

However I have just one problem with all this: the hooks in Listing 9 are dependent on the hook classes supporting a particular interface. One promising alternative is to vastly simplify the hook interface by having a single hook method to which we pass an appropriate context identifyer, like PROGRAM(). This leads to our last implementation topic, which is to use procedural hooks.

8. Separated pre and post-process hooks with a generic interface

Listing 10 shows a variant of Listing 9 that decouples the hook from its implementation.

DEFINE CLASS FullyHookedButton AS CommandButton
  FUNCTION CLICK
    IF oApp.HookMgr( PROGRAM())  && The "Pre" hook
      THISFORM.SomeMethod()
    ENDIF
ENDDEFINE
DEFINE CLASS MyForm AS FORM
  ADD OBJECT oCmd AS FullyHookedButton
  FUNCTION SomeMethod
    * << Your code here >>
    THIS.oApp.HookMgr( PROGRAM())  && The "Post" hook
  ENDFUNC
ENDDEFINE

Listing 10, a component with a hook before calling a Click implementation method. The Click implementation method contains its own hook at its conclusion. Note that the hook calls are made to a hook manager method which permits a clean and generic interface.

The key here is the clean and invariant call, THIS.oApp.HookMgr( PROGRAM()), which allows us to have a single hook method serve for all application hooks. Of course, considerable logic may be required within the oApp::HookMgr() method but, on the other hand, this would allow us to table-base the hook operations, which is infinitely more flexible than hard-coding hooks. This mechanism is similar to the one used internally by the VFP Class Browser. More on this in the next article.

Known uses

VFP Class Browser: The VFP Class Browser is a real hookfest. By using its AddIn() method, and by specifying a method to hook, you can permanently register custom behavior for a variety of events. Registering the add-in is persistent because the AddIn() method creates a record in BROWSER.DBF, which lives in your HOME() directory.

Markus Egger's PowerBrowser: The public-domain PowerBrowser tool is an example of how a fully hooked application like VFP's Class Browser can be considerably adapted.

The INTL Toolkit: The INTL Toolkit, which I wrote, uses hook objects throughout.

Visual FoxPro Example

Here is a simple example of a hook system. Here we have a form with a textbox and a combobox connected by hooks as shown in figure 6.

Some things of note in the sample code:

Hooks are chained. The combobox, for example, chains through a number of objects, first picking up Dropdown() and Interactivechange() behavior, followed by the Form's hooks, which in this case attach a 2-part Rightclick() behavior.

The Textbox's RightClick() is supplemented by the Form's RightClick() and that of the Form's hook, in this case some additional debug choices (full functionality not wired-in).

Note the GarbageCollect() procedures which are woven into the Release() processes to clean up these pointers. Without this, the form will never release without CLEAR ALL. In some versions of VFP, expect a GPF without this pointer clean-up.

Figure 6: The sample provided creates this system. The combobox hook chain starts is composed of two behavior objects before hooking to the form and its hook. The textbox hook chain is similarly extended by the form and its hook object.

*---------------------------------------------
*-- Sample for Hook Operation design pattern
*-- Copyright 1997 by Steven Black
*-- Version 1.0 May 22 1997
*---------------------------------------------
*-- Create a hookable form
oForm= Create("Frm")
*-- Describe what the user should do
oForm.Caption= "RightClick around to execute hooks"
*-- add a hookable textbox
oForm.AddObject("txt", "Txt")
*-- add a hookable combobox
oForm.AddObject("Cbo", "Cbo")
oForm.Cbo.Top= 30
*-- add dropdown and interactivechange behavior to
*-- combobox by using hooks
Local loTemp
loTemp= Create("DropDown")
oForm.Cbo.SetoHook( loTemp)
loTemp= Create("Interactivechange")
oForm.Cbo.SetoHook( loTemp)
Release loTemp
oForm.Cbo.ToolTipText= ;
  "Make a selection to execute hooks"
oForm.Cbo.StatusBarText= ;
  "Make a selection to execute hooks"
*-- Give all controls the Form's Right-click behavior
oForm.txt.SetoHook( oForm)
oForm.Cbo.SetoHook( oForm)
*-- add a general shared debug utility object
oForm.AddObject( "Debug", "DebugMenuHook")
oForm.Debug.Top= 60
*-- Give all controls the debug menu behavior
*-- (via form, which is already in the hook chain)
oForm.SetoHook( oForm.Debug)
*-- Show the works, though you wouldn't normally
*-- make the debug hook object visible
oForm.SetAll("Visible", .T.)
oForm.Show()
Read Events

**************************************************
* Abstract Hook Behavior Class
**************************************************
*-- Class: abstracthook
*-- ParentClass: label
*-- BaseClass: label
*-- Baseclass for hooks
*
Define Class abstracthook As Label
  Caption = "Hook"
  ohook = .Null.
  Width= 150
  BackColor= Rgb(0,255,0)
  Name = "abstracthook"
  Procedure Init
*-- Say something smart
  This.Caption= This.Class
  Endproc
  Procedure Release
*-- Release from memory.
  Release This
  Endproc

  Procedure GarbageCollect
*-- Clean up pointers
  This.ohook= .Null.
  Endproc
*=== Hooked Events and Methods ====
* SetoHook( o)
* RightClick()
* PopUpMenu()
* InteractiveChange()
* DropDown()
  Procedure SetoHook
*-- Automate the hook setting process. If a hook
*-- already exists, hook to it, thus adding to the
*-- hook chain.
  Parameters toPassed
  If Type( "THIS.oHook")= "O" And ;
      PEMSTATUS( This.ohook, "oHook", 5)
    This.ohook.SetoHook( toPassed)
  Else
    This.ohook= toPassed
  Endif
  Endproc
  Procedure RightClick
  If Type( "THIS.oHook")= "O" And ;
      PEMSTATUS( This.ohook, "RightClick", 5)

    This.ohook.RightClick
  Endif
  Release Popups Shortcut
  Define Popup Shortcut Shortcut Relative From Mrow(),Mcol()

  This.PopupMenu()
  If Cntbar( "Shortcut") > 0
    Activate Popup Shortcut
  Endif
  Endproc

  Procedure PopupMenu
*-- Delegate to the implementation chain
  If ! Isnull( This.ohook) And ;
      TYPE( "THIS.oHook") = "O" And ;
      PEMSTATUS( This.ohook, "PopUpMenu", 5)
    Return This.ohook.PopupMenu()
  Endif
  Endproc
*-- Occurs when the user changes the value of a
*-- control using the keyboard or the mouse.
  Procedure InteractiveChange
*-- Delegate to the implementation chain
  If ! Isnull( This.ohook) And ;
      TYPE( "THIS.oHook") = "O" And ;
      PEMSTATUS( This.ohook, "InteractiveChange", 5)
    Return This.ohook.InteractiveChange()
  Endif
  Endproc
*-- Occurs when the list portion of a ComboBox
*-- control is about to drop down after the drop-down ;
*-- arrow is clicked.
  Procedure DropDown
*-- Delegate to the implementation chain
  If ! Isnull( This.ohook) And ;
      TYPE( "THIS.oHook") = "O" And ;
      PEMSTATUS( This.ohook, "DropDown", 5)
    Return This.ohook.DropDown()
  Endif
  Endproc
Enddefine
**************************************************
* Developer's Debug Menu Behavior (partial)
**************************************************
*-- Class: debugmenuhook
*-- ParentClass: abstracthook
*-- BaseClass: label
*-- Hook that produces debug choices in the
*-- current shortcut menu
*
Define Class debugmenuhook As abstracthook
  Name = "debugmenuhook"
  Procedure PopupMenu
  DoDefault()
  Define Bar 1001 Of Shortcut Prompt "\-"
  Define Bar 1002 Of Shortcut Prompt "Debug" ;
    MESSAGE "Invokes the debug window"
  Define Bar 1003 Of Shortcut Prompt "\*  0
    Activate Popup Shortcut
  Endif
  Endproc
  Procedure PopupMenu
*-- Delegate to the implementation chain
  If ! Isnull( This.ohook) And ;
      TYPE( "THIS.oHook") = "O" And ;
      PEMSTATUS( This.ohook, "PopUpMenu", 5)
    Return This.ohook.PopupMenu()
  Endif
  Endproc
  Procedure DblClick
  If ! Isnull( This.ohook) And ;
      TYPE( "THIS.oHook") = "O" And ;
      PEMSTATUS( This.ohook, "DblClick", 5)
    This.ohook.DblClick
  Endif
  Endproc
  Procedure Click
  If ! Isnull( This.ohook) And ;
      TYPE( "THIS.oHook") = "O" And ;
      PEMSTATUS( This.ohook, "Click", 5)
    Return This.ohook.Click()
  Endif
  Endproc
  Procedure DropDown
  If ! Isnull( This.ohook) And ;
      TYPE( "THIS.oHook") = "O" And ;
      PEMSTATUS( This.ohook, "DropDown", 5)
    Return This.ohook.DropDown()
  Endif
  Endproc
  Procedure InteractiveChange
  If ! Isnull( This.ohook) And ;
      TYPE( "THIS.oHook") = "O" And ;
      PEMSTATUS( This.ohook, "InteractiveChange", 5)
    Return This.ohook.InteractiveChange()
  Endif
  Endproc
Enddefine
**************************************************
*-- Class: frm
*-- ParentClass: form
*-- BaseClass: form
*-- Form with Rightmouse Behavior
*
Define Class frm As Form
  DoCreate = .T.
  Caption = "Hooked Form"
  ohook = .Null.
  AutoCenter= .T.
  Name = "frm"
  ShowTips= .T.
  Procedure Release
*-- Release from memory
  This.GarbageCollect()
  Release This
  Endproc

  Procedure GarbageCollect
*-- Clean up pointers
*-- ... recursively
  Local lni
  For lni= 1 To This.ControlCount
    This.Controls[lni].GarbageCollect()
  Endfor
  This.ohook= .Null.
  Endproc
  Procedure QueryUnload
*-- Clean up first
  This.GarbageCollect()
  Endproc

  Procedure Destroy
  Clear Events
  Endproc

*=== Hooked Events and Methods ====
* SetoHook( o)
* RightClick()
* PopUpMenu()
* InteractiveChange()
* DropDown()
  Procedure SetoHook
*-- Automate the hook setting process. If a hook
*-- already exists, hook to it, thus adding to the
*-- hook chain.
  Lparameters txPassed
  If !Isnull( This.ohook)
    This.ohook.SetoHook( txPassed)
  Else
    This.ohook= txPassed
  Endif
  Endproc
  Procedure RightClick
*-- Define a shortcut
  Release Popups Shortcut
  Define Popup Shortcut Shortcut Relative ;
    FROM Mrow(),Mcol()
*-- Delegate to a shortcut specialty method
  This.PopupMenu()
*-- Activate the shortcut
  If Cntbar( "Shortcut") > 0
    Activate Popup Shortcut
  Endif
  Endproc
  Procedure PopupMenu
*-- 3 sample shortcut menu bars
  Define Bar 100 Of Shortcut Prompt "Properties" ;
    MESSAGE "Display the properties sheet"
  Define Bar 200 Of Shortcut Prompt "Builders" ;
    MESSAGE "Invoke custom builders"
  Define Bar 300 Of Shortcut Prompt "Code" ;
    MESSAGE "Edit underlying code"
*-- Delegate to the implementation chain
  If ! Isnull( This.ohook) And ;
      TYPE( "THIS.oHook") = "O" And ;
      PEMSTATUS( This.ohook, "PopUpMenu", 5)
    Return This.ohook.PopupMenu()
  Endif
  Endproc
  Procedure DropDown
*-- Occurs when the list portion of a ComboBox
*-- control is about to drop down after the
*-- drop-down arrow is clicked.
  If ! Isnull( This.ohook) And ;
      TYPE( "THIS.oHook") = "O" And ;
      PEMSTATUS( This.ohook, "DropDown", 5)
    Return This.ohook.DropDown()
  Endif
  Endproc
  Procedure InteractiveChange
*-- Occurs when the user changes the value ;
*-- of a control using the keyboard or the mouse.
*-- Fire pre-process hook
  If ! Isnull( This.ohook) And ;
      TYPE( "THIS.oHook") = "O" And ;
      PEMSTATUS( This.ohook, "InteractiveChange", 5)
    Return This.ohook.InteractiveChange()
  Endif
  Endproc
Enddefine
**************************************************
*-- Class: txt (c:\_workshop\hooks.vcx)
*-- ParentClass: textbox
*-- BaseClass: textbox
*-- Textbox with Rightmouse Behavior
*
Define Class txt As TextBox
  Height = 23
  Width = 100
*-- Hook reference
  ohook = .Null.
  Name = "txt"
  Procedure GarbageCollect
*-- Clean up pointers
  This.ohook= .Null.
  Endproc
*-- Releases from memory.
  Procedure Release
  Release This
  Endproc
  Procedure PopupMenu
  Define Bar _Med_cut Of Shortcut Prompt "Cu\*  "U"
    This.ohook.SetoHook( toPassed)
  Else
    This.ohook= toPassed
  Endif
  Endproc
  Procedure Click
  If ! Isnull( This.ohook)
    This.ohook.Click
  Endif
  Endproc
  Procedure RightClick
  Release Popups Shortcut
  Define Popup Shortcut Shortcut Relative ;
    FROM Mrow(),Mcol()
  This.PopupMenu()
  If Cntbar( "Shortcut") > 0
    Activate Popup Shortcut
  Endif
  Endproc
  Procedure DblClick
  If ! Isnull( This.ohook)
    This.ohook.DblClick
  Endif
  Endproc
Enddefine

When you rightclick over the textbox, control is passed to the hook, which invokes a context sensitive shortcut menu. Some other things to draw from this example:

The hooked objects and the hooks dont need to be of the same Baseclass. Hooks can be lightweight objects, like labels, lines, or even Relations (which are ultra-lightweight).

Hooks can be shared by more than one object. There is nothing stopping us from having all the controls on our form pointing to the same hook.

Conclusion

Classes with code embedded with hook operations are generally more flexible than those that arent. If you define hook operations consistently and predictably throughout your application, then its usually possible to attach new behavior to satisfy the needs of a particular instance without subclassing. Moreover, hook operations can be reckoned and bound at run time.

Managing VFP Class Libraries

Last updated: 19 September 2008

The more applications you develop, and the bigger the teams that work on them, and the more often the software is revised, the more class library management skills are important. Face it: you can't ship a large application without good class management. as program size increases, proper structure becomes exponentially more important. this session, for advanced visual foxpro developers, is all about being an effective class librarian, and working smart with visual and code class libraries. we'll also talk about the conceptual and physical management of classes and class libraries, how the VFP Class Browser and Component Gallery play complementary roles in class library management, and how library structures are related to component architecture. Time permitting, we'll also look at tools to aid in library engineering, documentation, information dissemination, and tactics for version control.

I'm writing this document to share some sensible practices for dealing with VFP Class Libraries. Be aware that if you work alone, or if you are working on a small project, what I suggest herein matters much less than if you are developing a larger system, and collaborating with several other developers. That's because experience suggests that the complexity of managing Visual FoxPro classes and class libraries varies with the size of the project, the intensity of development activities, the complexity of the project's class design, and on the number of developers involved.

Like the old saying goes, to build a 1-meter high tower, all you need is a few undamaged beer cans. But to build a 10-meter or 100-meter high tower, you need a plan, and you need to pay attention to process and structure. That's what this paper is about: how to structure your class libraries so they can scale, at least a little bit.

So, without further ado

Major Rule: Use your own classes as the basis for the controls in your applications.

Figure 1 illustrates a class hierarchy spawned directly from a VFP base class. This is a classic rookie mistake. The problem here is this hierarchy culminates with a read-only class, a Visual FoxPro built-in base class. To make a global change to the hierarchy you'll need change all the first-level subclasses since the culmination class is read-only, so many of the so-called benefits from using an inheritance hierarchy are largely lost.

Figure 1. A class hierarchy spawned directly from a VFP base class.

Compare Figure 1 with Figure 2, which shows a class hierarchy wherein the classes culminate with a class of your own making. To make a global change to the hierarchy you can make a change to the culmination class (class _Textbox, in Figure 2) and, as long as the subclasses are properly implemented, they will inherit its changes.

Figure 2. A class hierarchy spawned from your own base class.

Using your own base classes between your operational classes and the VFP read-only base classes gives a buffer between your classes and the VFP base classes, protecting the class against changes in VFP base class behavior. It also gives you a way to integrate your own standards into the roots of all your classes.

Minor Rule: Use insulation classes to protect you from version changes.

Arguably the classes provided by third parties should be considered de-facto read-only, even if you possess the source code. Third party classes can be unpredictably volatile. If you make code changes to original third-party sources, you have a problem: how do you reliably integrate your changes in the event of new versions? Moreover consider the parallel problem of how to manage both enterprise-wide and application-specific alterations, and not touch this third-party source.

Insulation layers (Figure 3), sometimes colloquially known as I-Layers, are an artificial culmination point that allow centralized access to framework and other volatile subclassable structures. Reading upwards in figure 3, application-specific classes can vary from project to project, and the insulation layer can normally serve simultaneously as an enterprise feature layer and as a framework isolation layer.

Figure 3. Insulation layers protect you from framework changes at the price of an extra inheritance layer.

Insulation layers are not without obvious expenses. Foremostly they add layers to your class hierarchies, which makes things like debugging more tedious.

If you create software for multiple client, consider creating a layer for each company. This only makes sense in the case where a software company will be working repeatedly for a group of customers. The basic structure is a follows:

VFP base classes

Your VFP insulation layer

Customer 1 classes

Application 1 classes

Application 2 classes

Customer 2 classes

Application 3 classes

Application 4 classes

So an insulation layer is a good architectural idea that comes at the cost of slightly deeper hierarchies. Now, if you're following along, you see that we soon have all these classes in all these layers! How best to deal with them?

Good Practice: Know the VFP Class Browser and Component Gallery, and understand what each does best.

A class library is a physical package of classes. For managing visual class libraries (VCXes) you use the Class Browser. But classes can -- and should -- be abstracted in a variety of other ways. For this we can use the Component Gallery, to organize them, be they VCXes or PRGs, along with all the other elements of our software. The Component Gallery is great for creating virtual and abstract groupings of related items in a way that personally suits a particular situation.

Separate white papers on using the Class Browser and Component Gallery are available for download from http://www.msdn.microsoft.com/vfoxpro.

Good Practice: Use Ken Levy's superclass tool; it lets you see exactly what your inheriting.

The deeper your class hierarchies, the more you'll appreciate tools like SuperCls.PRG, which now ships with Visual FoxPro. Supercls gives you the ability to see and edit superclass code while working in the class designer's code editing windows.

Good Practice: Go to Tools|Options|Field Mapping and set your own base classes as the defaults, this integrates your classes into VFP's IDE

You can use Tools/Options/Field Mapping to create a set of default classes that VFP will instance when dragging fields from the Data Environment. John Petersen has a utility that helps manage field mapping for different projects to different classlibs/classes. Alternately you can use the REGEDT32 program and save the key

HKCU\Software\Microsoft\VisualFoxPro\5.0\Options\IntelliDrop

to a file in each project folder. To switch project mappings all you have to do is double click the file in explorer.

Minor Rule: Minimize the number of classes stored in each class library.

Visual SourceSafe only lets one person check out a class library at the same time. Individual classes, be they stored in VCX or .PRG files, cannot be individually checked out. So checking out a class means checking out the library wherein the class lives.

Moreover the probability of finding that a particular library is not available for immediate exclusive use rises with the intensity of development, the number of concurrent developers, and with the number of classes in each library. These forces imply that you should, as much as possible, shoot for small class libraries.

Remember that Visual Class libraries, VCX files, are units of physical distribution. It's a good idea to use class libraries to package things that should be packaged together. If this is so, then checking out a class library should give you exclusive access to things that are functionally related. So when designing class libraries, think cohesion instead of type. In this sense, unless you work alone or in a very small team, having a class library for "buttons" or "forms" makes little sense.

Major Rule: Logically store your classes to minimize coupling.

At first glance segmenting classes into libraries appears simple: Keep first-level subclasses in one VCXes, all forms together in one VCX, cool controls together in another, with other VCXes for, say, composite classes, forms, toolbars, and so on. Everything is straightforward and, if anything, things are easy to find.

But the guiding principle of class library segmentation is coupling. Physically arrange your classes so that distributing and reusing them is not hampered by external dependencies. Classes that depend on each other should probably be stored together, or proximately in related class libraries.

Also, if you can avoid .H files, then do so. H files are a pain in Visual FoxPro.

Minor Rule: Don't store customized classes with "base classes" in the same library.

If you follow the general design rule that all classes should spawn from read/write culmination classes, then these culmination classes are required by all the classes in your application. Now to later reuse these classes, you'll want to minimize unnecessary baggage and detritus when referencing superclass class libraries. Do this by keeping superclass libraries very lean.

Consider also segmenting your base classes into visible and non-visible base class libraries, and other divisions as well. That way, if you need to reuse an abstract non-visual class somewhere else, you don't need to link-in the unrelated visual classes that just happen to be in the same package as its superclass.

Good Practice: Prefer short class names - _txt vs _textbox

Visual FoxPro's compiler just checks syntax. It doesn't check for proper use of things like class, field, and variable names. The longer your class names, the greater the probability of a typo.

Good Practice: Don't go overboard naming your classes according to their pedigree.

While it's generally a good idea to prefix or suffix class names according to a (or any) sensible naming convention, I don't recommend going overboard with hierarchical naming conventions. In other words, avoid naming classes like this:

Txt
txtLevel1
txtLevel1Level2
txtLevel1Level2Level3

The reason for this is evident the minute you start redefining classes, which invariably happens if your class design isn't perfect.

Good Practice: Know when to store classes in PRGs, and when to store them in VCXes

Pros of PRG class libraries:

• PRG file is a single package, VCX is two (counting the VCT).
• Using a PRG allows it to be compiled while read-only.
• Using a PRG allows multiple developers to work in the same class although care should be taken when
• merging during check-in as VSS merge dialog is less than clear about its intentions.
• Allows a developer to easily see the difference between different versions
• Supports conditional exclusion of methods (and properties) using compiler directives--very helpful in
• debugging by optionally removing an error handler.
• Can easily add comments to individual properties.

Cons of PRG class libraries:

• Makes class structure much harder to understand due to the lack of a property sheet and class browser.
• Many developers are less comfortable in code.
• Can't use any of the ClassBrowser services
• Automated parsing of VCXes is far, far easier than parsing PRG, so many advanced maintenance functions are much easier with VCXes.
• Some of these cons will be addressed in VFP 7, as suggested by the direction indicated in the preview we saw at DevCon.

Good Practice: Know the VFP metadata structures

VFP's VCX files are but DBFs by another name. In this light, all of Visual FoxPro's designers (class designer, form designer, menu designer, report writer, and project manager) are just data entry program to fill VCX, SCX, MNX, FRX, and PJX structures.

Sometimes the most efficient way of doing something is to open and edit the class library as a normal table. This means USE xxx.vcx EXCLUSIVE, then edit the contents of something like the ClassLoc field directly and close the file. Then execute 'COMPILE FORM xxxx' (if you are hacking a .SCX) or 'COMPILE CLASSLIB XXX' (if you are hacking a VCX).

Back the files up before you try this - if you make a mistake you could lose the whole structure.

A cool thing to do is write batch programs that can manipulate directories full of forms in one go. Something like the following code which re-pointers the class library location attribute for all the forms in a directory.

lnFormCount = ADIR( laForms, '*.SCX' )
FOR lnCnt = 1 TO lnFormCount
USE ( laForms[lnCnt] ) EXCLUSIVE
REPLACE ALL classloc WITH <newpath> ;
FOR classloc = <oldpath>
USE
COMPILE FORM ( laForms[lnCnt] )
NEXT

Major Rule: The "class" of a VFP composite class applies only to the underlying container.

Visual FoxPro supports a variety of containership abstractions. Formsets can contain forms, forms contain controls, grids contain columns which contain controls, optiongroups contain optionbuttons, and containers and custom classes can contain almost anything.

You can have, for example, an address visual class used to display addresses that contain textboxes and labels and perhaps some interactive behavior. But don't be fooled by appearances: The only pure classy thing about a container of objects is the container. The objects in the container are instances of other classes.

When you put a textbox class in another container class, you are not creating a subclass of the textbox, you are creating an instance of it. A subclass of your container will contain textboxes that inherit from your base textbox class, not from changes you make in the textboxes in the superclass container.

Read Drew Speedie's article, 'Dangers of Composite Classes in Development' (FoxPro Advisor, Feb 1998). The general rule is only create composite classes that you intend to instantiate, not that you intend to subclass.

Minor Rule: Use your own classes in your VFP grids and commandgroups.

Add your textbox classes to the grid columns and remove the FoxPro base textboxes. Just drag your textbox from the forms control toolbar into the column, then to eradicate the original text box by selecting the FoxPro base textbox and press delete.

Minor Rule: Understand when you should, and shouldn't, bundle VCXes in your app.

Do you have to distribute class libraries within executables? No! You can choose to exclude any file from a project and as long as it is along SET PATH then VFP will find it at run-time. Of course, you'll have to remember to distribute the proper files, but this strategy buys you great update and customization flexibility.

Major Rule: Use Source Control to Protect Your Class Libraries

Henceforth I talk about Visual Source Safe, but what I say applies to PVCS or any other leading source control systems you might be using.

Good Practice: If you are in experimental mode, coding from the seat of your pants, use the Source Safe safety net.

I often find that I would like to be able to copy a class so that I can experiment with it without committing any changes. This is the sort of thing Visual SourceSafe lets you do. You can, at any time, rollback to any previous version.

Guideline: Realize that Source Control Integration comes with baggage

Visual FoxPro comes with source code integration, but you don't have to use it that way. In fact many developers I respect eschew source control integration and use the source control products interactively. There are many reasons for this. Foremostly you'll notice that Source Safe gets progressively slower as its source database gets larger. Secondly the VFP integration provides few of the features and control available with using the Source Safe client interactively. Third, irritating Source Safe dialogs come up and clutter your VFP desktop, and forth the integration is managed with poorly documented intermediate files that are prone to being forgotten or erased as you copy files around in your development environment.

Guideline: Occasionally clean out your Source Safe project.

Source control is overhead. If you are using source control integration and you find that response time for opening a file is beginning to lag, consider cleaning out your source control database. I suspect, but can't be sure, that Source Safe response time varies inversely with the size of the Source Safe database, which increases every time you check something in.

Good Practice: Be aware of some common problems with Interactive Source Control.

If every time you open a project you get the message: "An unspecified registry error occurred". This means the project is controlled by Visual SourceSafe, but you don't have VSS activated in VFP, or don't have it installed at all. If you really don't need the SourceSafe link, you can clear the SCCDATA field in the PJX file and set the LOCAL field to .T.

Guideline: VFP needs Read/Write privileges in order to compile its structures.

If you have a project that does not allow building, and you get the error message "Cannot Write to Read-Only Cursor", usually this means a read only file. You probably have of your SCX or VCX files is still set to read-only but not marked as checked into SourceSafe in the project file.

In order to compile forms and classes, those files have to be read/write. This is because VFP stores the compiled code in the VCT or SCT. You probably don't have some VCX or SCX files checked out so they are read only on your drive.

If you are using Source Safe interactively, when you do a Get Latest Version on all files in the project (including tables) make sure you have 'Make Writeable' box option checked. VFP needs many of its structures to be writeable since object code is stored as part of its metadata.

VFP Design Pattern - Mediator

Last updated: 12 June 2008

A Mediator object is a form of Observer that abstracts inter-workings of two or more objects. They are communication hubs that serve to decouple objects — objects don’t need to know about each other, they just need to know their Mediator. Mediator promotes loose coupling by keeping objects from referencing to each other explicitly, and lets you vary their interaction independently.

Look for Mediator patterns where objects are decoupled and in situations when inter-object behavior variation is present or expected. Mediators well serve situations where complex protocols must be managed, and when centralized access points are desirable.

Mediator is a behavioral pattern. When we speak of Mediator, it’s a system of at least three objects messaging in a star topology.

Figure 1. The Mediator provides a common connection point, centralized (and subclassable) behavior and behavior management, all with a common interface. As the number of colleagues increases, the number of total communication pathways is vastly reduced.

Sample Mediator implementations in Visual FoxPro

The horizontal splitter control in Tazmanian Traders, which comes from the tsGen class library in the VFP samples, contains two shapes and two scrolling command buttons. As the handle shape is moved, or the scroll buttons are clicked, messages are sent to the splitter, which then messages selected items on the form.

In Codebook, toolbar navigation buttons call a business object class (cBizObj) which mediates in a one-way direction between the form (of class cBizObjForm) and the data behavior object (of class cDataBehavior). This simplifies the programming interface — only the cBizObj class protocols need be known — and allows for the easy substitution of the data behavior implementations from the cDatabehavior class. Note that several bridge patterns are evident here separating programming interfaces from implementations.

• cToolbarButton and cBizObjForm
• cBizObjForm and cBizObj
• cBizObj and cDataBehavior

Figure 2. Hitting the Next button in Codebook. The cBizObj class mediates between the form and the desired behavior.