====== A library for Forth ====== ===== The idea ===== Add a mechanism to Forth that allows code sections/chapters to be loaded from a large 'file' with a simple command. How the file is stored and what it contains is implementation-dependent. What matters is the interface. The sections or chapters are accessible through one or more keywords. The interface consists of the following word: ''%%NEED%%'' * ''%%NEED%%'' ''%%( i*x "name" -- j*x )%%'' Make sure “name” is present in the dictionary. If not, load it from a library. If "name" already exists, do nothing. An optional word as suggested by Ulrich Hoffmann * ''%%FROM%%'' ''%%( "name" -- i*x )%%'' Open the library "name" for use with ''%%NEED%%'' Optional words as suggested by Willem Ouwerkerk * ''%%NEEDED%%'' ''%%( i*x a b -- j*x )%%'' Make sure that name represented by the string a b is present in the dictionary. If not, load it from a library. If name already exists, do nothing. * ''%%RUN%%'' ''%%( i*x "name" -- j*x )%%'' Find "name" in the library and load & run the script from a library. * ''%%.LIB%%'' ''%%( -- )%%'' Show all section/chapter headers of the current library in one or more columns * ''%%VIEW%%'' ''%%( "name" -- )%%'' View the code from the libary, that belongs to "name". ===== Pseudo code ===== Function: NEEDED ( a +n -- i*x ) \ a +n is keyword a) Check if the word represented by the string a +n exists in the current search order. b) Do nothing the string was found, otherwise start searching the active library for a keyword that matches this string. c) When the keyword is found load the source code chapter otherwise issue an error message Function: NEED ( "name" -- i*x ) Perform the function of NEEDED using the parsed "name". Function: .LIB ( -- ) Print all chapter headers in a human readable form. Function: VIEW ( "name" --) When the keyword "name" is found view the source code chapter, otherwise issue an error message Function: FROM ( "name" -- ) Make the library "name" the current active library When "name" was not found issue an error message ==== Contributions ====

Implementations

===== A library for RP2040 Flash ===== Here, I will elaborate on an example that stores the source in Flash memory. For this implementation, there are the following design requirements:\\ - Simple structure ( idea A.N. ) - Quick search function - Extensible: OPEN-LIB CHAPTER CLOSE-LIB WIPE-LIB - Easy to use: NEED etc. ===== Structure ===== This library mechanism is based on two control characters ''%%09%%'' and ''%%0D%%''. They act as separators and are not part of the text itself. The ''%%09%%'' marks the end beginning of a chapter, like this:\\ Note that: The library starts with a dummy ''%%09%%'' so that the first keyword line can be found.\\ * ''%%09%%'' chapter-1 ''%%09%%'' chapter-2 ''%%09%%'' last-chapter ''%%09%%'' The ''%%0D%%'' marks the end of a line:\\ * line-1 ''%%0D%%'' line-2 ''%%0D%%'' last-line ''%%0D%%'' The first line of each library chapter starts with case insensitive keyword(s) separated by spaces. The word ''%%CHAPTER%%'' builds this the first line, and provides this line with a backslash to prevent the keywords for being executed while loading a chapter. Then it adds all source code until the ''%%'' marker is found. The end of each chapter is marked by ''%%09%%'' The end of the library is kept in a pointer that is updated when we extend the library.\\ chapter 2TUCK \ Copy top double below second : 2TUCK 2swap 2over ; ( x1 x2 x3 x4 -- x3 x4 x1 x2 x3 x4 ) %% chapter 2ROT \ Rotate third double to top of stack : 2ROT 2>r 2swap 2r> 2swap ; %% chapter -2ROT \ Rotate top double to third double position on the stack : -2ROT 2swap 2>r 2swap 2r> ; %% The above library chapters leave the following ASCII structure in Flash memory.\\ When after the <09> there is no more text, it's the end of the library.\\ \ 2TUCK<0D> \ Copy top double below second<0D> : 2TUCK 2swap 2over ; ( x1 x2 x3 x4 -- x3 x4 x1 x2 x3 x4 )<0D> <09> \ 2ROT<0D> \ Rotate third double to top of stack<0D> : 2ROT 2>r 2swap 2r> 2swap ;<0D> <09> \ -2ROT<0D> \ Rotate top double to third double position on the stack<0D> : -2ROT 2swap 2>r 2swap 2r> ;<0D> <09> ===== Pseudo code for the Flash library ===== Function: NEEDED ( i*x a +n -- j*x ) a) Save the string a +n and make the string uppercase b) If this string is already present in the dictionary, do nothing c) Start searching the library for this string d) When a keyword is found load the source code section until 09 is found e) When a keyword is not found issue an error message Function: NEED ( i*x "name" -- j*x ) a) Parse the next word form the input stream and perform the function of NEEDED Function: .LIB ( -- ) a) Scan the library for a 09 b) When it's not the last, print the found library chapter/section header ===== Pseudo code for expanding the Flash library ===== Function: OPEN-LIB ( -- ) Open the library by reading the last incomplete file sector to the sector buffer. Adapt the library write administration in such a way that new source code will be added behind whats read from Flash. When the library is empty, lay down the start of a section header with a 0D & 09 character. Function: CLOSE-LIB ( -- ) Close the library, writing the last added code (incomplete sector) Update the end of library pointer Function: WIPE-LIB ( -- ) Erase a stored library completely, restore the end of library pointer Function: CHAPTER ( -- ) Add a new library section, first add the section header. Note the given keywords in uppercase, sepatated by spaces and ended with a 0D. Read and store the Forth source code behind it until the end of source marker %% was found ===== noForth t library use ===== This sample is taken from the noForth t library mechanism. Here the library is stored in the remaining flash that is not needed by the noForth binaries. For readability some noForth specific words are replaced. create KEYWORD 20 allot \ Hold keyword 1008,1000 constant LIB \ Library start address LIB value LIBHERE \ Current end address of library : LIB-FIND ( a +n –- sa ) \ a,+n=name, sa=begin-source-section keyword place \ Save keyword keyword count upper \ Convert to uppercase libhere lib \ Library address range begin 2dup > 0= throw 2dup \ Nothing found? 0D scan nip over 1+ \ Get line with keywords, skip \ begin 2dup > while \ Line not done? bl skip dup >r \ Skip leading spaces bl scan \ After keyword r@ over r> - \ Keyword length keyword count s<> 0= \ Keyword found, leave source address if 2drop nip exit then repeat 2drop 09 scan 1+ \ Find library section, skip 09 again ; : LIB-REFILL ( source-id –- f ) to ib false \ Start of new line ib c@ 09 = ?exit \ End of source section, ready ib FF + ib 0D scan \ Find 0D (end of current line) dup ib - to #ib \ Calc. & save line length 1+ to source-id drop \ Save start of next line (behind 0D) >in ! true ; \ Refill succeeded : LIB-LOAD ( a –- i*x ) ?dup 0= ?exit \ Do nothing when zero @input >r 2>r \ Save current input source to source-id \ Set new input source #ib >in ! interpret \ Force a REFILL when loading source section 2r> r> !input ; \ ib #ib,>in@ source-id restore input source : NEEDED ( a +n –- i*x ) ['] lib-refill to 'refill \ Add library REFILL LIB-FIND keyword find nip 0= \ in the current forth search order and lib-load ; : NEED ( a +n –- i*x ) bl word count needed ; : .LIB ( –- ) \ Show library contents by printing keyword lines libhere lib \ Lib. range begin cr 20 for 1+ 1+ 2dup > 0= if \ Skip backslash, not past end of file? 2drop rdrop exit \ Ready then dup >r 0D scan \ Get end of keyword line r@ over r> - type \ Show keyword line 28 hor - 0 max spaces \ In fixed column size hor 28 > if cr then 09 scan 1+ \ Find section marker next key bl <> until 2drop ; ===== noForth t extending the library ===== Note that: This sample code uses calls to the built-in ROM API functions.\\ It uses them for reading, writing and erasing flash memory. 10000000 constant XIP \ Start of XIP memory create BUFFER 180 allot \ Sector buffer with overflow (noForth t) 0 value PTR \ Buffer index : LC, buffer ptr + c! incr ptr ; ( b -- ) \ Compile a byte in the library buffer : LM, bounds ?do i c@ lc, loop ; ( a +n -- ) \ Compile the string a +n in the library buffer : LIBWRITE ( +n -- ) \ Write library sector libhere xip - buffer 100 write-flash \ Write lib. sector to flash dup +to libhere negate +to ptr \ To next lib. block & correct pointer buffer 100 FF fill \ Erase first buffer buffer 100 + buffer ptr move ; \ Move overflow to sector buffer : BUFFER-FULL ( -- ) \ Buffer overflow, write & restore ptr FF > if 100 libwrite then ; : ADD-LIB ( text -- ) \ Add library section, save it when a buffer is full begin buffer-full \ Buffer overflow, write & restore refill drop \ Read new line ib 2 s" %%" s<> while \ No (%%) delimiter? ib #ib lm, 0D lc, \ Store line repeat refill drop ; \ Read next line : (-BL) ( -- ) parea bl skip nip ib - >in ! ; \ Skip leading spaces in IB : CHAPTER ( text -- ) \ Contruct new library section s" \ " lm, \ New lib. header begin (-bl) bl parse \ Find keywords ?dup while 2dup upper lm, bl lc, \ Store uppercase in buffer repeat drop 0D lc, \ Add formfeed when done add-lib 09 lc, ; : OPEN-LIB ( -- ) \ Open a lib. for writing, read incomplete lib. sector too! libhere lib <> if \ Lib. not empty? libhere 100 /mod 100 * \ Get previous sector & ptr length dup to libhere \ Correct lib. pointer & calc. XIP address buffer 100 move to ptr {w \ Read uncomplete sector to buffer, open flash then 0 to ptr {W ; \ First lib. entry, open flash : CLOSE-LIB ( -- ) \ Close library, save unfinshed buffer too buffer-full ptr libwrite W} ; \ Write last sector if any & close flash : WIPE-LIB ( -- ) \ Remove previous stored library from flash lib xip - \ Convert to start sector address libhere lib - FF000 and \ Convert to erase sector length, max. 1 Mbyte 1000 + {W wipe-flash W} \ Erase one sector extra lib to libhere ; \ And reset library pointer ===== A viewer for the library ===== This viewer shows 16 or less lines at a time. It stops at a <09> character, prints a divider line and waits for user input. When the spacebar was hit, it goes on displaying the next chapter. Any other key leaves the viewer. ( A library consists of 09 {tab}, 0D {cr} and ASCII chars ) ( ranging from blank to ~ others chars are not allowed ) \ Show the library source code of a chapter : .LINE ( a1 -- a2 ch ) begin c@+ dup BL < 0= while emit repeat ; : .DIVIDER ( -- ) cr 10 0 do ." -- " loop cr ; : LIBTYPE ( a -- ) begin 10 0 do \ Max. 16 lines at a time cr .line 09 = \ End of source chapter? if .divider leave then \ Show divider & ready loop dup libhere < while \ More library source to be done? key BL <> if drop exit then \ Yes, ask key, stop on non space repeat drop ; : VIEW ( "name" -- ) bl word count lib-find libtype ; ===== A sample library source ===== This examples shows how many different code parts can be used inside the library.\\ - Just a version message printing some strings in a script - Interpret some noForth commands as script adapting a pointer - The addition of some high level routines, the VIEW function - Again a script, but with numeric input from the stack, changing the noForth configuration. It includes a test of the changed part. open-lib chapter version versie vsn cr .( NEED version 0.32 ) cr .( Library version 0.22 ) cr .( Size is about 174 kBytes ) %% chapter RESTORE-LIB v: inside \ Restore LIBHERE in case it got lost lib hx 50000 + lib hx FF scan nip to libhere cr .( Libhere ) libhere u. cr .( Size ) libhere lib - dm . .( bytes ) v: fresh %% chapter VIEW ( A library consists of 09 {tab}, 0D {cr} and ASCII chars ) ( ranging from blank to ~ others chars are not allowed ) v: inside also definitions \ Show the library source code of a chapter : .LINE ( a1 -- a2 ch ) begin c@+ dup bl < 0= while emit repeat ; : .DIVIDER ( -- ) cr 10 for ." {} " next cr ; : LIBTYPE ( a -- ) begin 10 0 do \ Max. 16 lines at a time cr .line 09 = \ End of source chapter? if .divider leave then \ Show divider & ready loop dup libhere < while \ More library source to be done? key bl <> if drop exit then \ Yes, ask key, stop on non space repeat drop ; v: extra definitions : VIEW ( ccc -- ) bl-word count lib-find libtype ; v: fresh %% chapter PIN SQUERY ( GPIO -- ) \ Change GPIO pin for S? need [IF] dup dm 30 2 within [if] drop dm 24 [then] \ Invalid switch pin? 1 cfg 1+ c! \ GPIO-xx for S? 4 cfg @ abs 4 cfg ! \ Make sure to (re)start the second image config cr .( Test S? ) s? . %% chapter 12MHZ \ change clock frequency dm 12 0 cfg ! \ Set frequency in MHz 4 cfg @ abs 4 cfg ! \ Make sure to (re)start the second image config \ Test new configuration %% close-lib ===== The library is just bare ASCII ===== When displayed the library chapters look like this:\\ \ VERSION VERSIE VSN cr .( NEED version 0.33 ) cr .( Library version 0.22 ) cr .( Size is about 174 kBytes ) \ RESTORE-LIB v: inside \ Restore LIBHERE in case it got lost lib hx 50000 + lib hx FF scan nip to libhere cr .( Libhere ) libhere u. cr .( Size ) libhere lib - dm . .( bytes ) v: fresh \ VIEW ( A library consists of 09 {tab}, 0D {cr} and ASCII chars ) ( ranging from blank to ~ others chars are not allowed ) v: inside also definitions \ Show the library source code of a section : .LINE ( a1 -- a2 ch ) begin c@+ dup bl < 0= while emit repeat ; : .DIVIDER ( -- ) cr 10 for ." {} " next cr ; : LIBTYPE ( a -- ) begin 10 0 do \ Max. 16 lines at a time cr .line 09 = \ End of source section? if .divider leave then \ Show divider & ready loop dup libhere < while \ More library source to be done? key bl <> if drop exit then \ Yes, ask key, stop on non space repeat drop ; v: extra definitions : VIEW ( ccc -- ) bl-word count lib-find libtype ; v: fresh \ PIN SQUERY ( GPIO -- ) \ Change GPIO pin for S? need [IF] dup dm 30 2 within [if] drop dm 24 [then] \ Invalid switch pin? 1 cfg 1+ c! \ GPIO-xx for S? 4 cfg @ abs 4 cfg ! \ Make sure to (re)start the second image config cr .( Test S? ) s? . \ 12MHZ \ change clock frequency dm 12 0 cfg ! \ Set frequency in MHz 4 cfg @ abs 4 cfg ! \ Make sure to (re)start the second image config \ Test new configuration ==== A little extra ==== Loading a lot of library code in one go. it is used like this: ''%%need( -rot -2rot d- asm\ )%%''\\ Note that ''%%BL-WORD%%'' is a typical noForth word that uses ''%%REFILL%%'' inside, so it can load multiple lines of library words when you need too. v: inside : NEED( ( keyw-0 .. keyw-n -- ) begin bl-word count \ Read next keyword 2dup s" )" s<> \ Not the closing paren? while needed repeat \ Ok, perform NEEDED on the keyword 2drop ; \ Ready v: fresh ==== More ====

Other implementations