From: Ersek, Laszlo on
On Mon, 5 Apr 2010, ar0 wrote:

> I'm stuck on a system without root-priviledges. Thus if I want some
> program the admin doesn't want to install systemwide, I'll have to
> locally install it $HOME.

I do this regularly.


> Now, if I can simply build the program from source everything is fine. I
> configure with --prefix="$HOME" and add PATH=~/bin:"$PATH" to my
> ~/.bashrc. This way I get "my" versions of programs, even if another
> version may already be in the "rest" of PATH.

I recommend an /opt/vendor/package-version style hierarchy, or just
/opt/package-version style.

Create a directory called ~/installed and install everything under it,
keeping bin, man, doc, lib etc. directories separate. This makes it very
easy to uninstall a package (ie. rm -r ~/installed/package-version).


> But what if some program needs a libary which is either outdated or
> non-existant on the system? If I build that library from source and
> install it in $HOME, I still have to communicate its whereabouts to the
> programs that depend on that particular lib.

Here are the environment variables that I use on GNU/Linux systems.

- C_INCLUDE_PATH: makes -I hackery superfluous with gcc.
- CPLUS_INCLUDE_PATH: same for g++.

Normally you would put these in CPPFLAGS at configure time or similar.

- LD_LIBRARY_PATH: this has nothing to do with building and everything
with invoking a dynamically loaded executable. It specifies the search
path for libLIBRARY.so.MAJOR_VERSION symlinks.

- LIBRARY_PATH: specifies the search path for .so and .a files at link
time. Normally, you would pass these as -L options in LDFLAGS.

- INFOPATH, MANPATH, PKG_CONFIG_PATH: to reach the installed documentation
from the command line and to make pkgconfig files available.

The actual regular files containing the dso-s are installed like this:

Directory:
$HOME/installed/package-major.minor/lib/

regular file:
libLIBRARY.so.major.minor.patchlevel (eg. 1.0.5)

symlink used at runtime (by the dynamic loader):
libLIBRARY.so.major (eg. 1)

symlink used at build time:
libLIBRARY.so

The last symlink allows you to specify a default library version used for
linking newly built apps. The second to last symlink allows you to specify
a concrete version used with executables that link against a given major
version of the library. (Different major versions are binarily
incompatible. Minor versions within the same major version are binarily
compatible.)

This scheme allows you to keep multiple versions of the same library
installed.


> Up to now every program I built uses the "classic" ./configure && make
> combo, ie all use makefiles. So can I perhaps use a makefile with
> certain variable definitions which is then included automatically by all
> other makefiles?

If you set up the variables mentioned above correctly, you don't have to
alter any raw Makefile or pass any switches to configure, it will Just
Work (TM).


> If I could talk to the apropriate tools (gcc, ld), I would
> tell them:
> "Before you look for anything in the 'default' places, please
> first take a look in $HOME"
>
> Like, first look for headers in $HOME/include, then the default
> locations. If the library you need is already in $HOME/lib, use it.
> Etc.
>
> Is this possible in any convenient way?


Yes. The above still leaves you with the burden of maintaining those
environment variables manually, which is horrific. I used two solutions
for this until now.

One is (which works system-wide, ie. being root and installing everything
under /opt or /usr/local): write a shell script (or preferably a C
program) that traverses the whole tree and symlinks all files from under
the package specific directories to a few "unified" directories, ie.
/opt/pkg-major.minor/bin/* to /home/ar0/installed/bin. Same with lib,
libexec, info, man, doc, sbin and so on. A few extras can be considered
too, like automake files, pkgconfig files etc. You add the "unified"
directories /home/ar0/installed/{bin,lib,sbin,libexec,man,info} to the
corresponding env. vars and never touch the variables afterwars. If you
remove a package or install new ones, just re-run the "relink" program. If
you're interested, I'll try to find my implementation based on nftw() and
extended POSIX regular expressions.


The other solution (where the number of manually installed packages is not
very big and so the env. vars don't grow too long) looks something like
this (actual code):

1. I have a script called "$HOME/bin/scripts/pathsetup" on my PATH (script
not portable):

----v----
#!/bin/bash
set -e -C

function pathinit()
{
local PATHVAR="X$$_$1"
local INITVAL="$2"

eval "$PATHVAR=\"\$INITVAL\""
}

function pathprepend()
{
local PATHVAR="X$$_$1"
local DIR="$2"
local CURVAL=${!PATHVAR}
local CURVAL_EMB=":$CURVAL:"

if [[ "$CURVAL_EMB" != *:"$DIR":* ]] \
&& [ "/" = "${DIR:0:1}" -a -d "$DIR" -a -x "$DIR" ]; then
if [ ! -z "$CURVAL" -a ":" != "${CURVAL:0:1}" ]; then
DIR="$DIR:"
fi
eval "$PATHVAR=\"\$DIR\$CURVAL\""
fi
}


pathinit PATH /usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games
pathinit MANPATH "$(manpath -g -q)"
pathinit LD_LIBRARY_PATH /usr/local/lib:/usr/lib:/lib

pathprepend PATH "$HOME/bin/scripts"
pathprepend PATH "$HOME/bin/standalone"

for BASE in "$HOME"/installed/*; do
pathprepend CPLUS_INCLUDE_PATH "$BASE"/include
pathprepend C_INCLUDE_PATH "$BASE"/include
pathprepend INFOPATH "$BASE"/info
pathprepend LD_LIBRARY_PATH "$BASE"/lib
pathprepend LIBRARY_PATH "$BASE"/lib
pathprepend MANPATH "$BASE"/man
pathprepend PATH "$BASE"/bin
pathprepend PATH "$BASE"/libexec
pathprepend PATH "$BASE"/sbin
pathprepend PKG_CONFIG_PATH "$BASE"/lib/pkgconfig
done

rm -f ~/.path
set \
| sed -n "s/^X$$_/export /p" >~/.path
----^----


This creates a file ~/.path with contents like

----v----
export CPLUS_INCLUDE_PATH=...
export C_INCLUDE_PATH=...
export LD_LIBRARY_PATH=...
export LIBRARY_PATH=...
export MANPATH=...
export PATH=...
----^----

With the values used in the "pathinit" function calls placed at the end.


2. ~/.bashrc sources ~/.path; ~/.bash_profile sources ~/.bashrc


3. Whenever I install/remove a package, I re-run pathsetup and re-login.


HTH,
lacos
From: Ersek, Laszlo on
On Mon, 5 Apr 2010, William Ahern wrote:

> ./configure --prefix=[PATH] CPPFLAGS="$(cppflags)" LDFLAGS="$(ldflags)"
>
> It works just as well for non-autoconf builds--not all the world uses
> autoconf. Most builds already know to set the builtin linker paths, but
> if they don't then you also might need to add that to the LDFLAGS (e.g.
> "-Wl,-rpath /path/to/lib/"). You can make an `rpaths' shell function to
> generate those.

Great idea!

On a second thought, if you upgrade /usr/local/foo-major.minor to
foo-major.(minor+1) because of a newly fixed security vulnerability, won't
that break all executables searching for libFOO.so.major under the first
directory? Hm, you said /usr/local/[project], so I guess not. Cool.

Thanks,
lacos