Local Perl build with custom CPAN modules

Goal: build a Perl that is mostly identical to the Perl shipped with Mac OS X but is non-threaded to play better with Starlink modules and uses HPC Fortran (and C/C++, where feasible) to build the Math and Statistics modules.

  1. Download perl-73 from Mac OS X 10.7.3 open source archive
  2. Unpack the tarball and apply patches (10 total)
  3. Become root and execute the build via make
  4. Install the resulting 5.12 build:
    1. cd /tmp/perl/Build/5.12/DSTROOT/usr/local
      rsync -avE . /usr/local/
  5. Update /usr/local/Library/Perl/5.12/AppendToPath to add the system extra CPAN modules
  6. Upgrade CPAN modules to current versions
    1. Install /usr/local/Library/Perl/5.12/CPAN/Config.pm to configure changes to be installed in /usr/local/Library/Perl/Updates
    2. Run the upgrade command in the CPAN shell to update modules and install dependencies
    3. Remove any modules that are updated but have older versions that appear higher in the search tree or contain obsolete files (e.g. Tie::Watch, version::vpp)
  7. Update /usr/local/Library/Perl/5.12/CPAN/Config.pm to configure the installation of additiona modules in /usr/local/Library/Perl/Site
  8. Install all modules in the list of desired development tools (and dependencies) using the notest option. However, the following modules require special attention and should be installed first:
    • download CSS and patch lib/CSS/Parse/CompiledGrammar.pm before building and installing
    • manually download versions 0.16, 0.17, 0.18, and 0.19 of Test::YAML::Meta, unpacking each to /tmp and building/installing via /usr/local/bin/cpan Makefile.PL INSTALLDIRS=perl; make; make install
  9. Build and link required Homebrew packages; note that the order in which they are linked is important!
  10. Update /usr/local/Library/Perl/5.12/AppendToPath to add the Starlink CPAN modules
  11. Restart cpan after setting the following environment variables:
    • PATH=/usr/local/bin:/star/bin:$PATH
    • LD_LIBRARY_PATH=/usr/local/lib:/star/lib:/usr/X11/lib:$LD_LIBRARY_PATH
    • DYLD_LIBRARY_PATH=/usr/local/lib:/star/lib:/usr/X11/lib:$DYLD_LIBRARY_PATH
    • PKG_CONFIG_PATH=/usr/local/lib:/star/lib:$PKG_CONFIG_PATH
  12. Install all modules in the list of desired development tools (and dependencies) using the notest option. However, the following modules require special attention and should be installed first:
    • download Math::LP::Solve and obtain the source tarball from here before building and installing
    • download Math::Cephes and patch Makefile.PL before building and installing
    • download GD and patch Makefile.PL before building and installing
    • If the desire is to actually run the tests for the modules, the following patches are required:
      • Math::Currency: 0001-fixed-t-004_localize.t-for-MacOS.patch and 0002-fixed-t-002_basic.t-for-MacOS.patch from here
      • Astro::ADS: two patches from here
      • Astro::VO::VOEvent: here



0 comments

BibTeX to iTunes: Sync PDFs with iOS

Purpose

Import local PDFs referenced in a BibDesk library into iTunes for synchronization with iBooks. Creates 'track' entries in the iTunes database that contain information appropriately mapped from reference entries:
  • Article titles or book names become the title
  • Author lists map to Artist, and AAS-style citations (e.g. Alpher, Bethe, and Gamow 1948) are mapped to Album Artist
  • Journal & volume or conference proceedings title map to Album
  • Category, genre, and grouping populated per library, per journal, or per entry as needed

Setup

  1. Install the script file in ~/Library/iTunes/Scripts Download file "BibTeX to iTunes.scpt"
  2. Install the configuration plist file in ~/Library/Preferences Download file "net.bartonia.BibTeXtoiTunes.plist"
  3. Edit the plist file to match details of the reference library (or libraries) to be synchronized
    1. the dictionary Documents must contain at least one dictionary named after the library file that contains a dictionary listing the default category, genre, and grouping to apply to each entry within it; multiple libraries may be specified, and any of category, genre, or grouping may be omitted
    2. the dictionary Journals may contain one or more journal names, each of which references a dictionary containing any or all of category, genre, or grouping to apply by default to references from that journal
    3. the dictionary LaTeX abbreviations may contain one or more key-value pairs mapping LaTeX macros to the full names of journals they represent; if present, a journal-name value must match an entry in the Journals dictionary

Operation

Launch BibDesk, launch iTunes, then select BibTeX to ITunes from the script menu. Before running the script, unselect Copy files to iTunes Media folder when adding to library if you wish to keep the PDFs in their original location rather than moving them to the Books folder of the iTunes folder.

To specify the genre, category, and/or grouping for a particular entry (i.e. override the default values for the referenced journal or BibDesk library), place them in an AppleScript record structure in the Annote field of the reference, e.g.

{category:"foo",genre:"bar",grouping:"baz"}

Notes and TODOs

  • Neither category nor grouping, nor any other documented field, corresponds to the Collection feature of iBooks; after synchronization, documents will need to be manually moved into such Collections on the iOS device. Hopefully Apple will fix this issue in an upcoming iTunes release.
  • The allocation of data to the Artist and Album Artist fields looks proper in iTunes but the opposite choice would seem to work better in iBooks. Perhaps Apple will eliminate this distinction (or make it configurable) in a future release of iTunes (but I'm not counting on it).
  • The formatting of the 'citation keys' reflects a clear astronomy-literature bias but should probably be sensible for most people.
  • It would be handy to launch BibDesk automatically (and close it at the end of synchronization) if it isn't already running
  • It would be really nice to specify whether to move the PDF files via an item in the configuration plist and to toggle the iTunes preference appropriately

0 comments

AFP automounting and other Mac OS X Server admin stuff

While the manuals and knowledge-base articles at the main Mac OS X Server site contain much helpful information, I never really got things configured the way I wanted until this past weekend when I stumbled across two papers:

These two really put things in context and filled in some of the gaps from the official manuals. At long last I was able to:
  • set up the USB drives that hold music, pictures, and other shared data so that they now automount over AFP when, say, iTunes is launched
  • locate and enable the various bits of Workgroup Manager that allow proper assignment of policies (e.g. laptop sleep/wake schedule, Kerberos usage, VPN configuration) across multiple hosts at once...and just better understand how to use WM in general
  • force Home Sync to suppress conflict messages and just let the mobile (laptop) accounts win for background syncs
One quirk of automounting that was not covered caused me some grief but was eventually solved after some trial and error. When setting up automounted Share Points via Server Admin, the default is to have them show up under /Network/Servers/hostname/sharename. I wanted them to appear under /Volumes (or /Groups, in one case) to ensure that the paths were the same on all Mac hosts. Choosing the middle two options---to appear under /Network/Applications or /Network/Library---turns out to be a bad idea even if naming weren't an issue because Finder and various other applications (e.g. iPhoto) apparently presume that anything mounted there is read-only (strangely, Terminal and iTunes were able to pick up the shares as read-only). The best bet is to use the last opton, Custom Mount Point, and set the path to be the same as the original. However, after applying these settings I still intermittently had some connection issues---mostly in the form of read-only mounting---that I solved with the following bit of data massaging:
  1. Open Workgroup Manager
  2. Click the All Record Types button
  3. Choose Mounts from the drop-down menu
  4. In the mountOption and VFSOpts fields, edit the values to remove ;AUTH=NO%20USER%20AUTHENT@
  5. Save and exit (maybe restart AFP for good measure)
The default values written by Server Admin set up the automounts with guest authentication, which seems to be an issue particularly with Kerberos. Also, it is a good idea at this point to comment out the line
 /-            -static

in /etc/auto_master on the server; if that is not done, weird things will happen especially when one of the shared drives is unmounted (say for disk maintenance).


---

Also came across some helpful Kerberos stuff:



0 comments

Script to backdate blog entries

The wiki/blog server provided by Mac OS X Server does not provide any capability to alter the timestamps applied to entries, and while it nominally utilizes the MetaWeblog API it appears to completely ignore any timestamps supplied. This is kind of annoying, as it can be convenient to add time-related notes as entries after the fact.

After some hacking around to figure out where the various entry data are kept in the system, I put together a script that allows me to roll back the creation date (and subsequent update timestamps) for a particular entry:


Download file "change_blog_createDate"

  1. Create the entry
  2. After saving, note the unique id value assigned to it
  3. Run the script on the server
sudo change_blog_createDate --user <username> <unique_key> <date string>

The date string can be anything that would parse to a valid date via the Perl CPAN modules Date::Manip or DateTime::Format::DateManip; it does not need to be supplied as a single value, all its tokens are joined together by whitespace before parsing.


For a group blog, use

sudo change_blog_createDate --group <groupname> <unique_key> <date string>

0 comments

Shared calendars and wiki calendars post 10.6.4

Unsupported method of accessing Apple Wiki Server calendars from iCal client was broken by 10.6.4 upgrade as discussed here. My solution utilizing hard-linked directories is described here.

0 comments

Hard-linking directories on HFS+

Following up on discussions and examples here and here, I created two tools to create and remove hard links to directories on HFS+ filesystems. The code was copied from the discussion sites, but in keeping with the admonitions to be very, very careful in using such links I augmented the code to be a bit more precise. The code for creating a directory link is

#include <unistd.h>
#include <stdio.h>
#include <sys/stat.h>
int main(int argc, char *argv[]) {
if (argc != 3) {
fprintf(stderr,"Usage: hlink_dir <src_dir> <target_dir>\n");
return 1;
}
struct stat st;
stat(argv[1], &st);
if ( st.st_mode & S_IFDIR ) {
int ret = link(argv[1], argv[2]);
if (ret != 0) perror("link");
return ret;
} else {
fprintf(stderr,"Usage: hlink_dir <src_dir> <target_dir>\n");
fprintf(stderr,"<src_dir> must be a directory\n");
return 2;
}
}

which was turned into an executable via

gcc -o /usr/local/bin/hlink_dir hlink.c -Wall

and the code for the link remover is

#include <unistd.h>
#include <stdio.h>
#include <sys/stat.h>
int main(int argc, char *argv[]) {
if (argc != 2) {
fprintf(stderr,"Usage: unhlink_dir <target_dir>\n");
return 1;
}
struct stat st;
stat(argv[1], &st);
if ( st.st_mode & S_IFDIR ) {
int ret = unlink(argv[1]);
if (ret != 0) perror("link");
return ret;
} else {
fprintf(stderr,"Usage: unhlink_dir <target_dir>\n");
fprintf(stderr,"<target_dir> must be a directory\n");
return 2;
}
}

which was similarly turned to an executable via

gcc -o /usr/local/bin/unhlink_dir

0 comments

Setting up customized LDAP schema

Followed instructions from Zytrax, OpenLDAP, Linux Gazette, OAV, and a few others to come up with a customized schema and slapd.conf changes to utilize it:

Download file "bartonia.schema"
Download file "slapd.conf.diff"
Download file "slapd_macosxserver.conf.diff"

Initially followed the instructions from AFP548 to drop the schema into place and restart, but that didn't work.Discovered that it was best to use ldaptest to process the configurations and generate a new configuration directory (which the OpenLDAP documentation says is preferred over config files now). The procedure to use these files (presumed to be dropped into /tmp) and update the directory is:


launchctl unload /System/Library/LaunchDaemons/org.openldap.slapd.plist
cd /etc/openldap
cp /tmp/bartonia.schema schema/
for file in slapd.conf slapd_macosxserver.conf
do
mv $file $file.apple
cp -p $file.apple $file.bartonia
patch </tmp/$file.diff
ln -s $file.bartonia $file
done
mv slapd.d slapd.d.apple
mkdir slapd.d.bartonia
chmod go-rx slapd.d.bartonia
slaptest -v -f slapd.conf -F slapd.d.bartonia
ln -s slapd.d.bartonia slapd.d
slapindex -v
launchctl load /System/Library/LaunchDaemons/org.openldap.slapd.plist

(All of the above commands were run as root, either via sudo or by using sudo to open a root shell.)

Loading of entries could commence once the top-level domain and ou entries were added.

0 comments

Astronomy and Math CPAN modules

  1. Added the following lines to AppendToPath
    1. /usr/local/lib/perl5
    2. /star/Perl/lib/perl5/site_perl
  2. Created a custom CPAN configuration in /usr/local/CPAN that sets INSTALL_BASE=/usr/local. Installations and updates of these modules must be proceded by "cd /usr/local; /usr/local/bin/cpan"

0 comments

MacTeX

Installed MacTeX. Ran Tex Live Utility immediately afterwards to apply updates. Done.

0 comments

Scisoft and SAOImage for IRAF

Scisoft was downloaded and installed, primarily because it makes setting up IRAF pretty painless. SAOImage was installed afterwards since Scisoft includes an older version.

Tweaks:
  • ln -s /Applications/scisoft/all/Packages /Applications/scisoft/all/packages (needed to avoid some case-sensitivity issues)
  • link and copy various paths in /Applications/scisoft/all/Packages/iraf/iraf/local to /Users/iraf
Finally, set up IRAF

  1. cd /Applications/scisoft/i386/Packages/x11iraf-1.5DEV
  2. ./install

0 comments

HPC gcc/gfortran & Starlink via Homebrew

Homebrew formulae were cooked up for HPC gcc and gfortran as well as Starlink too.

0 comments

Homebrew for Open Source / Public Domain package management

The Homebrew tool has been installed for wrangling Open Source and Public Domain software packages:

  1. curl -L http://github.com/mxcl/homebrew/tarball/master | tar xz --strip 1 -C /usr/local
  2. cd /usr/local
  3. brew install git
  4. git init
  5. git remote add origin git://github.com/mxcl/homebrew.git
  6. git pull origin master
Formulae are to be updated from time to time via

  1. cd /usr/local
  2. git pull origin master

0 comments

Mac OS X-ish Perl 5.10.1

Build Perl 5.10.1 that looks mostly like the 5.10.0 that ships with Mac OS X.

Starting points:
After some trial-and-error hacking, I figured out what files were being patched and extended in order to build things in the Apple Way. The main adjustments are to tweak the INC path search order and add the ability to use AppendToPath and PrependToPath to make further adjustments to it. The resulting files were captured in the attachment.
Download file "make_localperl_5.10.1.tar.gz"

The build process is:
  1. tar xzf perl-5.10.1.tar.gz -C /tmp
  2. cd /tmp/perl-5.10.1.tar.gz
  3. tar xzf make_localperl_5.10.1.tar.gz
  4. make
  5. make install
Afterwards, the CPAN configuration file was set up:

Download file "Config.pm"
and the AppendToPath file edited:

Download file "AppendToPath"

and then the cpan tool was used to upgrade and add modules.
Download file "Snapshot_2010_02_25_00.pm"

0 comments