apricoti

PerlModuleDependencyTracking

Page history last edited by Jonathan Buhacoff 3 yrs ago

I'm interested in generating a graph of the dependencies in my code.

I don't expect any module to be able to detect dynamic dependencies that result from random user input at runtime. But given some code that uses dynamic dependencies and given a test environment or test inputs, I expect to be able to track from beginning to end of the test which modules have been used.

Most of the tools I found were for end-users to calculate dependencies of packages they want to install. Accordingly, most of the tools rely on the dependency information included with the packages. Some of them scan sources to generate a list of dependencies, but they are pretty much limited to the obvious "use" and "require" statements that appear in the source. One of them scans the compiled bytecode, which I think gives it an advantage since it doesn't have to recurse when looking for dependencies (perl does that for free) but I think it has the same limitation.

The situation I want to improve is for a developer who has lots of modules in a project (not all my own) and not all of them are included with "use" and "require" statements. Some of the dependencies are only known at runtime based on some inputs. However, the range of inputs may be predictable. Even where the range of inputs is not predictable, some of the more common inputs may be predictable enough to set them up as test cases which can then be evaluated for dependencies.

Implementation ideas

Because its for developers, I think it makes sense to take advantage of the build process. Specifically, using the testsuite to evaluate the package dependencies. It might work like this:

Tracking files

1. Run a script that initializes the dependency database for the project. It might be a test named 00-depend.t, or it might be a command executed by the makefile's "test" target just before executing the tests. The current time is written to the database.

2. Run whatever tests exist for the module.

3. Run a second script that examines the database (now containing a record and order of files which were accessed during the tests) and generates a dependency list or tree. Or maybe this second script examines all the files in the hard drive to find files which were accessed after the recorded initial time (a command already exists for this part) and then generates the list or tree.

Hooking @INC

If the application has a small amount of entry points, one could place a hook into @INC that records from whence it was called and which module is being sought, and then lets the rest of @INC take care of it. The record can then be examined after the application exits.

Alternatives

I searched for perl dependency graphers and I got a few hits, but none of them really seemed to fit my style I guess:

http://www.cs.virginia.edu/~nrp3d/work/dependency.html

This one doesn't appear to be finished, but anyway from the description I can tell they are focusing first on plainly visible "use" and "require" statements, and only later will they tackle run-time dependencies.

http://depfind.sourceforge.net/Manual.html

Looks great, but it's for Java. What's interesting about it is that they examine the compiled bytecode and not the source. I think it can be implemented in perl/parrot.

http://search.cpan.org/~adamk/Algorithm-Dependency-1.101/

General-purpose dependency algorithm. It appears to be written with installers in mind although the documentation states specifically it's not very good for installers because it can't produce an ordered schedule. It could be suitable for generating the list of dependencies for my components, because I'd have to pass that as data to any dependency tracker. But I'm not so enthusiastic about finding depdencies in my code by myself. Not that my code is complicated... it's just the principle of the thing, I guess.

http://search.cpan.org/~rclamp/Module-Depends-0.10/

Has two implementations. One uses META.yaml files that are supposed to be included in perl module distributions. Not for me because I want to find my dependencies so I can generate such a file. The second implementation builds distributions in a fake environment and intercepts calls to makefile utilities. Also not for me, for the same reason.

http://search.cpan.org/~pkent/Module-Dependency-1.13/

Sounds like it only searches for "use" and "require" words in the modules. Also, there's a weird bit in the documentation about the reverse graph only being updated as files are searched... meaning if they are not found alphabetically in order of dependency then the reverse graph won't be complete??? I can't imagine they would settle for that. I guess I should try it and see.

http://search.cpan.org/~nuffin/Algorithm-Dependency-Objects-0.01/

An object oriented implementation of Algorithm-Dependency

http://search.cpan.org/~lembark/Schedule-Depend-2.6/

This is more for job execution. I don't think I can use it at all.

http://search.cpan.org/~saper/CPAN-Dependency-0.09/

Uses info packaged in perl module distributions and in CPAN database. It's more for working with already known information.

http://search.cpan.org/~zoccav/PerlBean-1.0/lib/PerlBean/Dependency.pm

I don't know.

http://search.cpan.org/~rbs/Slay-Maker-0.01/

I think it requires data to be handed to it.

http://search.cpan.org/~msergeant/AxKit-1.6.2/install/ExtUtils/AutoInstall.pm

Requires data to be handed to it. Maybe also uses dependency info packaged in perl module distributions.

http://search.cpan.org/~mschilli/CPAN-Unwind-0.03/lib/CPAN/Unwind.pm

Uses Module::Depends::Intrusive, which I already determined isn't really for me.

http://search.cpan.org/~autrijus/Module-ScanDeps-0.51/

Documentation says it does not track dynamic dependencies. Looks like a great module otherwise.

http://search.cpan.org/~gyepi/Ovid-0.06/

Intended to convert CPAN packages to RPM packages. I don't have proof but I bet it relies on the dependency information included in the CPAN packages.

http://search.cpan.org/~atourbin/rpm-build-perl-0.5.2/lib/B/PerlReq.pm

Extracts dependencies from perl modules and scripts by compiling them and then looking at the bytecode. Sounds nifty, but I don't think it solves dynamic dependencies.

http://search.cpan.org/~holt/makepp-1.19/pod/makepp_scanning.pod

Looks like it's not for perl.

http://www.flourish.org/cinclude2dot/

For C/C++

http://www.gfdl.noaa.gov/~vb/mkmf.html

Fortran and C/C++