• Bytesmiths Editions -- large, archival, fine-art photography on unusual materials
• Bytesmiths Press -- artists' services: web design/hosting, jury slides, giclee reproductions, opening announcements, brochures, etc.
• Champagne Beadworks -- handcrafted jewelry and beadwork
• Crafted By Carol -- handcrafted jewelry and beadwork
• EcoReality, an organization devoted to establishing a sustainable ecovillage
• Ecovillage Newsletter -- Diana Leafe Christian's news of her travels.
• Environmental Education Outreach -- providing environmental education worldwide.
• Gemini Gypsy -- Carole Good-Hanson's fused glass frames
• Green Chipper -- light forestry and environmental services.
• Salt Spring Island Society for Community Education -- community education on our island of 10,000.
• Veggie Van Gogh -- two artists' mobile warehouse and living quarters, petroleum-free!
• Veggiemog -- life and times of Kelly O'Toole's Unimog, running on biodiesel

In the June issue, we made a case for "continuous documentation," and outlined what that entails. We also promised to give you some concrete examples and source code, so you could begin to implement a continuous documentation process.

There are at least five widely differing Smalltalk dialects out there, augmented by two major and numerous minor code management systems. Rather than attempt the impossible task of embracing such diversity, we're presenting stuff that is actually implemented and working in VisualWorks® 2.0 under ENVY/Developer® R1.43.

Many of these things can be done in other environments. However, much of the following assumes you can associate storage with arbitrary software components, which might be difficult if your code manager is simply a layer on top of source code files.

• Principle 1: Conceptual Integrity -- Documentation must be at the same level as that which it describes. There is simply no way you can cram all your documentation needs into class and method specifications. ENVY provides nestable modules called Applications and SubApplications that have a specification field, and a module binding component called a "configuration map" that also has a specification field.
• Principle 2: Constant Accuracy -- Documentation must be stored with that which it describes. If you want your developers to maintain their documents, you've got to make it easy for them to do so.
• Principle 3: Accessibility -- Documentation must be available quickly and efficiently from other, related documentation. This does not mean sending a reference number via email to your organization's technical library!

A Simple Macro Facility

The marriage of these three principles demands some way of linking related parts of documentation together. VisualWorks has a simple but efficient tagged character class, Text , that allows you to associate arbitrary objects with each character in a string. (If your Smalltalk has no such thing, you will need to either add it, or come up with some other linking mechanism.)

This tagged character capability suggests a simple "not quite hypertext" linking facility. [951223-JWS: See also our marketing blurb on a much-improved full hypertext implementation that is available with our consulting services.] First off, we need to fix a bug; add the following instance method to TextStream :

TextStream :

  nextPutAll: aText
 
    "Place each of the elements of aText on myself, starting at my
 current position. If I'm fed an instance of Text, keep its emphasis.
 Answer aText."
 
    (aText respondsTo: #runs)
      ifFalse: [^super nextPutAll: aText].
 
    1 to: aText size do: [:i |
      "I know, it's brute-force. Someday, this should be optimized by
       examining runs."
      self
        emphasis: (aText emphasisAt: i);
        nextPut: (aText string at: i)].
    ^aText

Without this bugfix, Text fed to a TextStream loses its emphasis, which sorta defeats the purpose! Readers of our last column may recognize a "signature testing" pattern of managing system changes, and may recall that an override is indeed a base image change. Be aware that code that expects the superclass behavior will now be "broken" by this bugfix!

Now implement the following three methods in a class extension of Text . We try to put system additions in a parallel application with a similar name. In our repository, the following extensions are in a subapp called DevelopmentBytesmiths , since they relate to the development process, and not particularly to Text in its full generality. Note also that these extensions are dependent on Compiler ; simply adding these extensions to the subapp Collections , where Text is defined, would create a circular dependency between the apps Kernel and Compilation .

Text :

  withInclusions
    "Answer a copy of me in which all strings with the emphasis
  #Smalltalk are evaluated and replaced with a String or Text
  representation of the result."
 
    ^self withInclusionsIn: nil
 
  withInclusionsIn: codeOwner
    "Answer a copy of me in which all strings with the emphasis
  #Smalltalk are evaluated and replaced with a String or Text
  representation of the result. The context of evaluation is the
  object codeOwner."
 
    ^(self
      withInclusionsIn: codeOwner
      on: (TextStream on: (String new: self size * 2
        "Guess that inclusions may double size."))) contents
 
  withInclusionsIn: codeOwner on: textStream
    "Place on textStream a copy of me in which all strings with the
  emphasis #Smalltalk are evaluated and replaced with a String or Text
  representation of the result. The context of evaluation is the object
  codeOwner. Answer textStream."
 
    | whereAmI whereWasI |
    ^(self size > 0 and: [runs values includes: #Smalltalk])
      ifFalse: [textStream nextPutAll: self]
      ifTrue:
        [whereAmI := whereWasI := 1.
        [whereAmI := whereAmI + (self runLengthFor: whereAmI).
        (self emphasisAt: whereWasI) == #Smalltalk
          ifFalse: [textStream nextPutAll: (self copyFrom: whereWasI to: whereAmI-1)]
          ifTrue: [(Object errorSignal
            handle: [:ex |
              textStream
                emphasis: #bold;
                nextPutAll: '*** Cannot include the following expression!! ***';
                emphasis: nil;
                cr;
                nextPutAll: (string copyFrom: whereWasI to: whereAmI-1).
              ex returnWith: '']
            do: [Compiler
              silentEvaluate: (self copyFrom: whereWasI to: whereAmI-1)
              for: codeOwner
              logged: false]) withInclusionsIn: codeOwner on: textStream].
        whereWasI := whereAmI.
        whereAmI = 0 or: [whereAmI > self size]] whileFalse: [].
        textStream]

The VisualWorks 2.0 compiler is in many ways as old as Smalltalk itself, and has not been fully integrated with signals and exceptions. We fixed that by adding silentEvaluate:for:logged: , which always raises an exception when compilation breaks. You can do the same, or you can change this to evaluate:for:logged: and put up with the occasional syntax error window when expanding Text that has bad expressions to expand.

This facility is recursive; if a section of Text has the emphasis #Smalltalk, the resulting expansion is itself scanned for inclusions, so any object that understands withInclusionsIn:on: can implement its own expansion scheme. In fact, without someone putting an end to this recursion, there can be real trouble! Implement the following two methods in extensions to Object and String , respectively:

Object :

  withInclusionsIn: ignored on: textStream
    "Place on textStream a printable representation of me suitable for
  use in documentation. Answer textStream.
    This is a 'bug catch' message in this class, and should normally
  only be sent to Texts or Strings. Subclasses should not normally
  override simply to change presentation."
 
    self printOn: textStream.
    ^textStream

String :

  withInclusionsIn: ignored on: textStream
    "Place myself on textStream for use in documentation. Answer
  textStream."
 
    textStream nextPutAll: self.
    ^textStream

For even more flexibility, add the following to an extension of BlockClosure . If the Text being expanded is a block, the block will be evaluated, and the result will be inserted into the expanded output. The block can optionally take the "code owner" and the active TextStream as arguments, which allows included source code blocks to query their environment and directly manipulate the resulting expanded output.

BlockClosure :

  withInclusionsIn: codeOwner on: textStream
    "Place on textStream a printable representation of my evaluation
  suitable for use in documentation. Depending on the number of
  arguments I take, pass me codeOwner and textStream on evaluation.
  Answer textStream."
 
    | args |
    args := Array new: self numArgs.
    1 <= args size ifTrue: [args at: 1 put: codeOwner].
    2 <= args size ifTrue: [args at: 2 put: textStream].
    ^(self valueWithArguments: args)
      withInclusionsIn: codeOwner
      on: textStream

These methods add the basic "hypertext" behavior to Text . Now your app/subapp comments can have things like MyClass comment with the emphasis #Smalltalk, and sending withInclusions to that Text will embed the comment for MyClass . But by itself, this inclusion facility is not terribly useful for two reasons:

1. ENVY strips the per-character attributes from Text before storing it.
2. Basic VisualWorks provides no user interface for applying custom character attributes to a Text .

ENVY Text storage

If your hypertext goes away when your image quits, it won't improve your team's productivity one bit! When you press the "source" button in an ENVY browser to switch to "comment" mode, any changes you make are stored as Strings in "inherited user fields." These are arbitrary key value storage locations, in which both key and value must be a String . If we can trick ENVY into storing Text (or even arbitrary objects) in these fields, it saves us from changing each of the many places where these fields are accessed.

This gets a little difficult, because the source code for methods that access the repository has been removed, and your ENVY/Developer license keeps you from decompiling or otherwise reverse engineering those methods. The following technique allows you to copy the hidden methods and associate that copy with a new method selector, allowing you to provide an original implementation in its place that conditionally sends the old method.

OTI has no legal objections to this technique, but it can be dangerous if misused! We've been using the following for some time, but if there is a typo, or if you use this technique to intercept and modify other hidden methods, you may damage your ENVY repository. Be sure to follow our suggestions in last month's column for managing base image changes, and consider making and testing these changes in a separate repository until you are certain they are safe.

Do this in a workspace to copy and rename the two hidden methods that need to be intercepted:

  | meth oldRecord |
  meth := (UserFieldRecord compiledMethodAt: #contents) copy.
  oldRecord := meth record.
  meth selector: #contentsFromVendor.
  UserFieldRecord
    updateEditionsRecordIn: LibraryManagement
    with: [:record | record
      addMethod: meth
      source: nil
      basedOn: oldRecord
      changeCategoryTo: 'intercepted methods']
    ifUnable: [].
  meth := (Record class compiledMethodAt: #libraryFormatFor:) copy.
  oldRecord := meth record.
  meth selector: #libraryFormatForFromVendor:.
  Record class
    updateEditionsRecordIn: LibraryManagement
    with: [:record | record
      addMethod: meth
      source: nil
      basedOn: oldRecord
      changeCategoryTo: 'intercepted methods']
    ifUnable: []

Before we go any further, we need to establish the predicate we are going to use for switching between the original implementation and our Text capable implementation. Put the following two methods in the same Application or SubApplication where you put the other class extensions:

Text class:

  canReadFrom: chars
    "Does the String (or streamed String) chars contain information
  that can be used to create an instance of me via #readFrom:?"
 
    | signatureChars |
    ^chars size >= "String new asText storeString size" 56 and:
      ['(Text string: '''
        occursIn: (chars isSequenceable
          ifTrue: [chars]
          ifFalse:
            [signatureChars := chars next: "'(Text string: ''' size" 15.
            chars position: chars position - 15.
            signatureChars])
        at: 1]

Text :

  libraryFormat
    "Answer a representation of myself suitable for storing in ENVY
  user fields."
 
    self storeString

Now replace those two hidden methods that we copied with original implementations that conditionally send the renamed method:

UserFieldRecord :

  contents
    "Answer my contents, decoding them if necessary."
 
    | charStream decoder |
    charStream := self collection readStream position: self startPosition - 1.
    ^((Text respondsTo: #canReadFrom:) and: [Text canReadFrom: charStream])
      ifTrue: [Text readFrom: charStream]
      ifFalse: [self contentsFromVendor]

Record class:

  libraryFormatFor: anObject
    "Answer a format suitable for storing anObject in the library."
 
    ^(anObject respondsTo: #libraryFormat)
      ifFalse: [self libraryFormatForFromVendor: anObject]
      ifTrue: [anObject libraryFormat]

Notice the conditional nature of these changes. These changes can safely replace those in the base image, because they will not break if the Application or SubApplication containing the Text extensions is not present.

With these changes, character emphases that you change in any comment or notes field will be preserved in ENVY. More importantly, the newly emphasized commentary does not "break" if these changes are not present; since they are in the storeString format, they are a simply a bit difficult to read.

You now have all the "modelling" changes you need to implement hypertext based on arbitrary Smalltalk expressions embedded in Text objects -- that's always the hardest thing to get right. Now all you need is a UI!

Setting #Smalltalk Text Emphasis

We've added extensive support for viewing, searching, and modifying these embedded expressions; a magazine column format cannot do such things justice. However, there are a few simple things you can do to get started, which may be enough to fill your needs, or it may inspire you to greater accomplishments.

Figure 1. An application comment template, showing hyper-links.

ParagraphEditor is the class that generates and modifies Text , including character emphasis. Unfortunately, there is no simple way to fully support new emphasis types without changing existing ParagraphEditor code, making new TextAttributes and CharacterAttributes instances, and then managing those instances as long lived system resources.

We're going to cheat by adding a simple facility for setting and removing the new #Smalltalk emphasis we've specified in a less general way. To be able to set or remove an arbitrary emphasis, add the following to an extension of ParagraphEditor :

ParagraphEditor :

  addEmphasis: emphasis
    "Add the given emphasis to the emphasis of the current selection."
 
    | thisText |
    thisText := self selectionStartIndex = self selectionStopIndex
      ifTrue: [Text string: 'x' emphasis: emphasisHere]
      ifFalse: [self selection].
    thisText addEmphasis: emphasis
      removeEmphasis: #()
      allowDuplicates: false.
    self selectionStartIndex = self selectionStopIndex
      ifTrue: [emphasisHere := thisText emphasisAt: 1]
      ifFalse: [self replaceSelectionWith: thisText].
    view topComponent refresh
 
  removeEmphasis: emphasis
    "Remove the given emphasis from the emphasis of the current
  selection."
 
    | thisText |
    thisText := self selectionStartIndex = self selectionStopIndex
      ifTrue: [Text string: 'x' emphasis: emphasisHere]
      ifFalse: [self selection].
    thisText addEmphasis: #()
      removeEmphasis: emphasis
      allowDuplicates: false.
    self selectionStartIndex = self selectionStopIndex
      ifTrue: [emphasisHere := thisText emphasisAt: 1]
      ifFalse: [self replaceSelectionWith: thisText].
    view topComponent refresh

Now, wire it to a pair of "hot" keys. This method works like the other emphasis hot keys, such as "ESC b" to add bold and "ESC B" to remove bold, but unlike the method that implements other emphasis changes, this one is not hard wired to a particular key.

ParagraphEditor :

  changeInclusionKey: aChar 
    "Add or remove 'inclusion' emphasis of the current selection, depending on the 
case of aChar. Uppercase removes, lowercase adds."
 
    aChar keyValue isUppercase
      ifTrue: [self removeEmphasis: #(Smalltalk)]
      ifFalse: [self addEmphasis: #(Smalltalk)].
    ^true

Finally, evaluate the following statements to bind this emphasis change to a "hot" key of your choice. (We chose "h" for "hyper," and because it was not already used.) These statements should go in the loaded method of the Application or SubApplication where you have been putting all this code. (While you're at it, add a removing method that undoes these key bindings.)

  (ParagraphEditor classPool at: #Keyboard)
    bindValue: #changeInclusionKey: to: ESC followedBy: $h;
    bindValue: #changeInclusionKey: to: ESC followedBy: $H.
  ParagraphEditor withAllSubclasses do: [:peClass |
    peClass allInstancesDo: [:pe |
      pe flushKeyboardMap]]

Viewing The Results

Now you are able to create and store run time inclusions in your project documentation. This allows detailed documentation to be linked to abstract documentation, yet be maintained at the detailed level, while not distracting the reader of the abstract information. But what if the reader wants to be so distracted? How do we see this stuff?

The simple way is to select a #Smalltalk-emphasized expression and inspect it. You get a basic inspector on a Text , which allows you to see the plain ASCII expansion. This suggests a simple, yet effective alternative: implement an Inspector subclass for Text that allows you a WYSIWYG view of Text .

Rather than use up precious column space with an entire implementation, we'll tease you with some salient bits:

  Inspector subclass: #TextInspector
    instanceVariableNames: ' '
    classVariableNames: ''
    poolDictionaries: ''
    category: 'BaseToolsBytesmiths'
 
  TextInspector comment: 'This class allows normal ParagraphEditor
    style editing of a Text just by inspecting it. It has no list view,
    since you don''''t really need one.'

TextInspector class:

  view: anInspector in: area of: superView
    "Create a text view on anInspector in area of superView."
 
    superView
      add: 
        (LookPreferences edgeDecorator on:
          (anInspector class textViewClass
            on: anInspector 
            aspect: #text
            change: #acceptText:from:
            menu: #textMenu initialSelection: nil))
      in: area

The rest is eight methods that all override Inspector methods.

This works great as the "developer" interface -- your developers will be so glad to be rid of keeping class comments in synch with app/subapp comments, that they won't mind that their hyperlink to embedded documentation is an inspector. But something better is needed for the "linear reader" and for hardcopy.

Unfortunately, any presentation change that is available from existing browsers is going to be a base image change. We considered wiring inclusion expansion to a new ParagraphEditor menu item, but we didn't feel the facility was general enough, and we certainly didn't want it slipping through to the end user!

Instead, we dug into the EnvyBrowser hierarchy, adding yet another mode to the code pane, with a menu item in the status area to expand all inclusions. This isn't the entire story, but it should get you started:

AbstractMethodsBrowser :

  expandComment
    "I assume that my text view is in comment mode. Cause the comment
  to be re-generated, with all Text of #Smalltalk style evaluated and
  inserted in-place."
 
    | oldState |
    [oldState := self textSelector.
    self changedTextTo: #textShowingCommentWithInclusions]
      valueNowOrOnUnwindDo: [self textSelector: oldState]
 
  textShowingCommentWithInclusions
    "Answer the expanded comment."
 
    ^Cursor read showWhile: [self commentString asText withInclusions]

Figure 2. Example of fully expanded documentation.

Conclusion

Getting project documentation to flow out of the development process can greatly increase productivity, and if it is well done, the developers actually begin to enjoy producing and maintaining their documentation!

In two columns, we've outlined the requirements and principles of a continuous documentation process, and provided some concrete examples of how it can be accomplished. In the future, we'll periodically revisit the topic with further design sketches and examples.