Articles
Searchable List of ArticlesList Alphabetically All Articles
How to build Harbour apps using pCode DLL with hbmk2
 Author: Eric Lendvai     
Table of Contents
Introduction
Republication of message from Przemyslaw Czerpak
Introduction
This entire article is a republication of a message/explanation Przemysław Czerpak did in Feb 2010 about how to compile Harbour applications when also creating and using Harbour build dlls that are compiled in pCode mode (default method).
Przemysław Czerpak is one of the core contributors to Harbour.

I found the email at https://www.mail-archive.com/harbour@harbour-project.org/msg26049.html

Some clarifications and concepts.

Harbour programs can be created is such manner that the C generated code relies on pCode, a series of C arrays and Symbol tables. 
It is also possible to create Harbour code that do not rely on pCode.
The pCode (portable code machine ) method relies on a Virtual Machine. The generated EXEs and DLLs are smaller and only slightly slower (20% worst case scenario) than pure C code base compilation.
The Harbour pCode IS NOT like VFP fxp, C#, Python or Java bytecode . Harbour's pCode is still pure C, meaning extremely fast.

1. There are two methods to include the Virtual Machine and runtime functions (core Harbour functions), including everything in your EXE (non shared mode), or linked in shared mode with harbour*-dll supporting files.
2. There are two methods to use Harbour DLLs, statically linked to your main EXE, or like most Windows DLLs dynamically (where the acronym is from "dynamic link library").

The term DLL is not properly used here, since it could be linked statically. 

Since Harbour also runs on Linux, iOd (Based indirectly from BSD) and any platform supporting C, here is an explanation of DLLs:
DLL - An external software module used by the Linux operating system. The official name for this type of module is a "shared object" or "shared library," but due to the universal embrace of Windows, which uses DLLs, the DLL term occasionally creeps into Linux lingo.

So between the two methods in 1. and 2., we now have 4 (2*2) solutions for building EXEs that also use DLLs. 
Republication of message from Przemyslaw Czerpak
Let's consider on MinGW only. Other C compilers are very similar but they cannot use .DLL libraries directly and need import libraries so such library have to be created together with PCODE DLLs using -implib hbmk2 option.


1. Main application is linked in shared mode with harbour*-dll using -shared hbmk2 switch. It is suggested method to use PCODE DLL.

In such case everything is trivial and works out of the box without any hacks and modifications in the code. i.e.:

/*** dllcode.prg ***/
proc MY_PROC()
? PROCNAME() + ": hello !!!"
return
init proc start
? "INIT PROC", PROCNAME()
return
exit proc end
? "EXIT PROC", PROCNAME()
return


/*** maincode.prg ***/
proc main()
? "start"
my_proc()
? "end"
?
return


To create PCODE dll simply use: hbmk2 -shared -strip -hbdyn dllcode.prg -n -w -es2
and dllcode.dll is generated, if you are using other then MinGW C compiler then you should add -cflag=-DHB_DYNLIB (to force exporting public functions inside .prg code - only MinGW supports auto export feature) and -implib switches (to create import library - only MinGW supports direct DLL usage by linker),

To compile and link final application use: hbmk2 -shared -strip -L. -ldllcode maincode.prg -n -w -es2

Please note that both DLL and final code are linked with harbour*.dll (-shared switch) and to link final application I added -L. and -ldllcode so linker was able to find dllcode.dll in current directory and MY_PROC() function inside.

As you can see all works without any problems and you do not need any tricks and/or hacks and/or additional code. You do not have to even touch original code.

It's possible because linker was able to cleanly resolve all references at link time. No manual hacks which can only cause unexpected crash.
In this version code inside PCODE DLL can access all Harbour exported functions (in case of MinGW all public functions when automatic export feature is enabled).


2. Main application is linked in shared mode with harbour*-dll using -shared hbmk2 switch but PCODE dll is not linked statically at link time but rather loaded dynamically by HB_LIBLOAD().

/*** dllcode.prg ***/
DYNAMIC QOUT, PROCNAME // added to inform compiler and linker that
// references to above functions have to be
// resolved dynamically at runtime
proc MY_PROC()
? PROCNAME() + ": hello !!!"
return
init proc start
? "INIT PROC", PROCNAME()
return
exit proc end
? "EXIT PROC", PROCNAME()
return


/*** maincode.prg ***/
DYNAMIC MY_PROC // declare functions from dynamically loaded DLLs
// as DYNAMIC
proc main()
local hLib
? "start"
hLib := hb_libload( "dllcode" )
if empty( hLib )
? "Cannot load library."
else
my_proc()
hb_libfree( hLib )
endif
? "end"
?
return


To create PCODE dll use: hbmk2 -nohblib -strip -hbdyn -lhbmaindllp dllcode.prg -n -w -es2

Please note that I added hbmaindllp which is linked with final PCODE DLL binaries and used -nohblib to not link any Harbour libraries.
With this switch I'm sure that I do not link the same core code twice with PCODE DLL and with final binaries and if I forget to declare some functions as DYNAMIC then I will have link time error so I can easy add missing declarations.

To compile and link final application use: hbmk2 -shared -strip maincode.prg -n -w -es2

Please note that in this version I haven't added -ldllcode because now it will be loaded dynamically by hb_libload() and linker does not have to know about it.
And that's all. You can test maincode.exe to see how it works.
This version needs to declare functions called from external modules as DYNAMIC to inform compiler and linker that they are bound dynamically at runtime so it should not create any static references and special code from hbmaindllp linked with PCODE DLL.
But as you can see it also does not need any link time hacks. All can be compiled and linked cleanly without even single warning.
If you have big application with a lot of functions in the core part and you want to use them in PCODE DLLs then you can make your life easier and automatically generate .ch file with all such functions using this code in your main application:

#if 1 // uncomment it for tests */
proc main()
genFuncList()
return
#endif


proc genFuncList()
local aFunc, hFile
aFunc := getFuncList()
asort( aFunc )
hFile := fcreate("dynamic.ch")
aeval( aFunc, {|x| fwrite( hFile, "DYNAMIC " + x + hb_osNewLine() ) } )
fclose( hFile )
return


func getFuncList()
local aFunc, nCount := __dynsCount(), nDst:=0, n
aFunc := array( nCount )
for n := 1 to nCount
if __dynsIsFun( n )
aFunc[ ++nDst ] := __dynsGetName( n )
endif
next
asize( aFunc, nDst )
return aFunc


It generates "dynamic.ch" file with DYNAMIC declaration for all functions registered in HVM. Then in your .prg files used to create PCODE DLLs you can simply add:
#include "dynamic.ch"
and you will not have to declare them manually.


3. Main application is linked in static mode with Harbour static libraries and PCODE DLL is linked statically just like in point 1.

Such version needs Harbour static libraries compiled with -DHB_DYNLIB so if you want to use it then you have to recompile Harbour core code using:
set HB_USER_CFLAGS=-DHB_DYNLIB
This condition can be eliminated by adding small extension to existing core code and HBMK2.

The test code is the same as in point 1 above.

To create PCODE dll use: hbmk2 -nohblib -strip -hbdyn -lhbmaindllp -cflag=-DHB_DYNLIB dllcode.prg -n -w -es2

Now -cflag=-DHB_DYNLIB is necessary also for MinGW because using hbmaindll disables auto export feature in MinGW.

To compile and link final application use: hbmk2 -static -strip -L. -ldllcode maincode.prg -n -w -es2

And that's all. Please only remember to recompile Harbour to create HVM lib which exports hb_vmExecute() and hb_vmPorcessSymbols() or the final binaries will not work. You will see only
start
end
messages on the screen.


4. Main application is linked in static mode with harbour static libraries but PCODE dll is not linked statically at link time but rather loaded dynamically by HB_LIBLOAD() just like in point 2.

Such version also needs Harbour static libraries compiled with -DHB_DYNLIB - see point 3 for above.

The test code is the same as in point 2.
To create PCODE dll use: hbmk2 -nohblib -strip -hbdyn -lhbmaindllp -cflag=-DHB_DYNLIB dllcode.prg -n -w -es2

To compile and link final application use: hbmk2 -static -strip maincode.prg -n -w -es2

and that's all.


The rest of the email are questions and answers on the topic


> It seems to work like this using Blinker, and it offered yet the
> option to replace or not to replace a doubly defined symbol when
> loading the DLL, but we couldn't implement it unfortunatelly.

Forget about link time function overloading. Modern C compilers
can inline code at different levels, i.e. during compilation and
in such case it's not possible to decompile compiled modules,
cleanly extract the code and insert different one in the same
place. In general such operation is forbidden and modern compiler
reports it as link time error.

> That is not my case of use - we use harbour self contained
> executable + pCode dll.

See versions 3 and 4 in above description.

> In xHarbour 0.9 it seems that hb_vmProcessSymbols and hb_vmExecute
> were declared inside maindllp.c but just as wrappers to
> (VM_DLL_EXECUTE) macro. Is this part of the same metodology you
> described?

They are declared in [hb]maindllp in all versions of Harbour and
[x]Harbour but these are only wrappers to real functions. MAINDLLP
is nothing more then manually written import library. It can work
_ONLY_ when corresponding symbols in harbour*.dll or in main EXE file
are exproted so they are visible for GetProcAddress() function.

If you are using statically linked application then it's necessary
to recompile Harbour using with HB_DYNLIB macro to force exporting
hb_vmProcessSymbols() and hb_vmExecute() functions. You can make it
by setting HB_USER_CFLAGS envvar, i.e.:

set HB_USER_CFLAGS=-DHB_DYNLIB

before compiling harbour code.
As I wrote above it's possible to eliminate this condition by
small modification in HVM library and hbmk2.

In xHarbour it is __EXPORT__ macro and C_USR envvar though in recent
version some symbols are always exported but it disables auto export
functionality so it does not work in xHarbour MinGW builds.

> >We still do not support HRL (libraries for HRB files what in
> >some cases can seriously reduce HRB functionality) anyhow HRB
> >is portable and always preferred format which has also many
> >features which are not available for DLLs.
> This HRL support could eliminate the need to use DLLs in my case. Is
> it planned to be done? How would it reduce HRB funcionality?

It's possible but if you haven't used HRB so far then it's
big chance that you haven't understood me correctly and you
are asking for sth what you do not know how to use. DLL does
not have such functionality so if you lived without it so far
then I do not understand why you need it to migrate from DLL
to HRB. I think that you should try to use HRB files 1-st.

Please note that you can store them anywhere. I.e. in your
own ZIP file or some other compressed file which you can
automatically decompress and load at runtime. I.e. use this
tool to collect all HRB files in current directory into
single mylib.dat file:

proc main()
local aFiles, file
aFiles := directory( "*.hrb" )
for each file in aFiles
file := hb_memoread( file[ 1 ] )
next
hb_memowrit( "mylib.dat", hb_zCompress( hb_serialize( aFiles ) ) )
return


and then load it at runtime using this function:

func loadmylib( cFile )
local cBody, aHRB, hrb
cBody := hb_memoread( cFile )
if !empty( cFile )
aHRB := hb_deserialize( hb_zUncompress( cBody ) )
for each hrb in aHRB
hrb := hb_hrbLoad( hrb )
next
endif
return aHRB


Please remember that it returns array with handles to HRB files.
When this array is destroyed then all loaded HRB files are automatically
unloaded so you should store it some variable as long as you need them
Then it's enough to assign NIL to this variable to force automatic
unloading. Of course you can extend above code and add some error
checking, password encryption, etc. All of the above can be done
in few lines using Harbour core functions.

best regards,
Przemek