[Pvfs2-developers] compiling statecomp

[Sam Lang]

On Feb 16, 2007, at 5:39 PM, Pete Wyckoff wrote:

> I was poking around in statecomp and became frustrated that it's
> difficult to compile, and that dependencies don't work properly.
> And that the way it is compiled is to lump all the .c files together
> to form a single executable in one go.  It sort of lives outside our
> nice Make infrastructure.
>
> Part of the reason may stem from the BG work in which it was noticed
> that statecomp should get built for the bulid machine, not the host
> machine, when cross-compiling.  But maybe that is fixable if one
> uses BUILD_CC and BUILD_LD as defined from configure?

I think a while back we changed the make dist process to not  
include .c files, but still generate the sources for statecomp from  
the lex/yacc files.  So if necessary, we should still be able to  
generate the statecomp sources from one machine, and not require flex/ 
bison elsewhere on the deployment machine.  I'm not sure this is the  
BGL issue you're describing though...

>
> This patch tries to fix things, but needs a look-over.  It does:
>
> 1. Compile statecomp as part of the top-level build, using BUILD_*.
>
> 2. Remove statecomp target, just type "make src/common/statecomp/ 
> statecomp"
>    if you really want to do just that one file, else let the
>    dependencies do their jobs.

Reducing the number of targets in the Makefile seems appropriate if  
it clarifies things.

>
> 3. Remove dependency of "%.c: %.sm" on statecomp.  This causes
>    massive rebuilds everytime you recompile statecomp, but is kind
>    of misleading I think.  We don't force rebuilds if the Makefile
>    or module.mk.in change, or if the gcc version changes.  Building
>    statecomp is for the hard-core who know how to test their changes
>    by rebuilding generated .c files by hand.

The few times I've ever changed statecomp it was to change the  
generated as well, so I would have always wanted to have this  
dependency.  Though doing make clean for the few cases where  
statecomp gets changed isn't a big deal either.  What are you  
changing in statecomp that doesn't result in changes to the generated  
files?  Just restructuring and cleanup?

What happens if I don't compile everything first, including  
statecomp, and try to modify a .sm and then try to get the .c  
generated by trying to make that one .c target?  Won't the thing barf  
because the statecomp binary doesn't exist?  Honestly, I can't  
imagine this ever happening in practice, and since its the only  
counter-example I could think of, I guess this change is ok.

-sam

>
> Thoughts?
>
> 		-- Pete
>
>
> Index: Makefile.in
> ===================================================================
> RCS file: /projects/cvsroot/pvfs2/Makefile.in,v
> retrieving revision 1.236
> diff -u -p -r1.236 Makefile.in
> --- Makefile.in	14 Feb 2007 21:51:57 -0000	1.236
> +++ Makefile.in	16 Feb 2007 23:31:07 -0000
> @@ -11,7 +11,6 @@
>  # publish        copy over documents to the PVFS.org web pags
>  # admintools     builds admin tools
>  # kernapps       builds userland helper programs for kernel driver
> -# statecomp      builds only the statecomp component
>  # cscope         generates information for cscope utility
>  # tags           generates tags file for use by editors
>  # codecheck      checks source code for nonconformance to our std.
> @@ -107,7 +106,7 @@ build_static = @build_static@
>  #    of a file that has the same name as the target.  Listing them
>  #    here keeps make from accidentally doing too much work (see GNU
>  #    make manual).
> -.PHONY: all clean dist distclean docs docsclean publish cscope  
> tags codecheck statecomp admintools kernapps
> +.PHONY: all clean dist distclean docs docsclean publish cscope  
> tags codecheck admintools kernapps
>
>  ################################################################
>  # Find project subdirectories
> @@ -498,11 +497,10 @@ VISMISCDEPENDS := $(patsubst %.c,%.d, $(
>  KARMAOBJS := $(patsubst %.c,%.o, $(filter %.c,$(KARMASRC)))
>  KARMADEPENDS := $(patsubst %.c,%.d, $(filter %.c,$(KARMASRC)))
>
> -# state machine generation tool
> -# cannot build statecomp objects with the cross compiler, as we
> -# build and run the tool on the build host at compile-time
> -#STATECOMPOBJS := $(patsubst %.c,%.o,$(STATECOMPSRC))
> -#STATECOMPDEPS := $(patsubst %.c,%.d,$(STATECOMPSRC))
> +# state machine generation tool, built for the build machine, not the
> +# host machine, in the case of cross-compilation
> +STATECOMPOBJS := $(patsubst %.c,%.o,$(STATECOMPSRC))
> +STATECOMPDEPS := $(patsubst %.c,%.d,$(STATECOMPSRC))
>
>  # DOCSPDF, DOCSPS, and DOCSHTML are lists of documentation files  
> generated
>  #   from latex
> @@ -691,11 +689,14 @@ $(ADMINTOOLS_SERVER): %: %.o $(LIBRARIES
>  $(KERNAPPS): %: %.o $(LIBRARIES)
>  $(KERNAPPSTHR): %: %.o $(LIBRARIES_THREADED)
>
> -# target for building _just_ the statecomp tool
> -#statecomp: $(STATECOMP)
> -#$(STATECOMP): $(STATECOMPOBJS)
> -#	$(Q) "  LD		$@"
> -#	$(E)$(LD) -o $@ $(LDFLAGS) $(STATECOMPOBJS)
> +# special rules to build state machine compiler using build host  
> compiler
> +$(STATECOMPOBJS): %.o: %.c
> +	$(Q) "  BUILD_CC	$@"
> +	$(E)$(BUILD_CC) $(CFLAGS) $< -c -o $@ $(call modcflags,$<)
> +
> +$(STATECOMP): $(STATECOMPOBJS)
> +	$(Q) "  BUILD_LD	$@"
> +	$(E)$(BUILD_LD) -o $@ $(LDFLAGS) $(STATECOMPOBJS) $(call  
> modldflags,$<)
>
>  # rule for generating cscope information
>  cscope:
> @@ -795,7 +796,7 @@ endif
>  # NOTE: we wrap this in ifneq's in order to prevent the
>  #    dependencies from being generated for special targets that don't
>  #    require them
> -ifeq (,$(filter statecomp clean distclean dist docs cscope tags  
> nodep,$(MAKECMDGOALS)))
> +ifeq (,$(filter clean distclean dist docs cscope tags nodep,$ 
> (MAKECMDGOALS)))
>  -include $(DEPENDS)
>  endif
>  # add this as a make goal to disable rebuilding dependencies
> @@ -820,8 +821,7 @@ nodep:; @:
>  	$(srcdir)/maint/pvfs2htmlfixup.sh $(@D)/*/$(@F)
>
>  # rule for automatically generated source files
> -# NOTE: this rule is disabled in release snapshots
> -%.c: %.sm src/common/statecomp/statecomp
> +%.c: %.sm
>  	$(Q) "  SMC		$@"
>  	$(E)src/common/statecomp/statecomp $< $@
>
> Index: src/common/statecomp/module.mk.in
> ===================================================================
> RCS file: /projects/cvsroot/pvfs2/src/common/statecomp/module.mk.in,v
> retrieving revision 1.12
> diff -u -p -r1.12 module.mk.in
> --- src/common/statecomp/module.mk.in	9 Apr 2005 19:26:45 -0000	1.12
> +++ src/common/statecomp/module.mk.in	16 Feb 2007 23:31:08 -0000
> @@ -15,11 +15,6 @@ STATECOMPGEN := \
>      $(DIR)/parser.c \
>      $(DIR)/parser.h \
>
> -# Trying to build lots of .c files at once confuses the MODCFLAGS  
> calculation.
> -# Point it to the target to get the real dir name.
> -statecomp: $(STATECOMP)
> -$(STATECOMP): $(STATECOMPSRC)
> -	$(Q) "  CC		$@"
> -	$(E)$(BUILD_CC) $(LIBCFLAGS) $(CFLAGS) $(call modcflags, $@) $^ - 
> o $@
> +MODCFLAGS_$(DIR) := -I$(DIR)
>
>  .SECONDARY: $(STATECOMPGEN)
> _______________________________________________
> Pvfs2-developers mailing list
> Pvfs2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>