Modern Perl Books, a Modern Perl Blog

Module::Install exists because installing CPAN distributions is not always perfectly easy.

Unfortunately, it didn't help—at least, not entirely. According to the completely unscientific process by which I install CPAN distributions, Module::Install accounts for a greater amount of pain than it should, at least according to its frequency of use. (Again, this is completely unscientific. I could guess that half of the CPAN client sessions which encounter Module::Install require me to fix things manually, but it's probably closer to 20%. It's more memorable because of my severe dislike for M::I prompting to install dependencies during configuration time.)

M::I addresses a real bootstrapping problem. I want to be able to use libraries during configuration, building, testing, and installation. I don't know which versions of those libraries you have available. Bundling known-good versions of those libraries with the distribution itself solves part of that problem...

... except when it doesn't. If I were to use M::I, I would have to re-release all of my distributions for every new release of every bundled library, at least if they contain important bug fixes for the various platforms about which I care. The cheap perfume of static linking leaves its musk heavy in the air.

It's easy to fall into the trap of a false dilemma. "You fool!" you prepare to comment below. "It's either that or the chaos of trying to make do with whatever version of those dependencies users may or may not have installed on their systems!" You're right; those are two possibilities. They're not the only two possibilities.

Part of the real problem is that bootstrapping during configuration is much too late. By the time you're running the configuration system, you're already running the configuration system. If your version of the configuration system is too old or too new, you have a problem. Bail out? Revert? Upgrade? There's no good heuristic for determining this. (The CPAN itself has an opinion. That's part of the problem.)

M::I hackers do deserve credit for helping to develop the META.yml standard. (I think M::I is the wrong approach, but I intend no slight toward its users, advocates, and developers. Invention requires the courage to get things wrong sometimes, even as it requires the courage to abandon false leads.) The META.yml specification is a big step in the right direction. If most CPAN modules have static requirements and follow a standard set of conventions, there's little or no configuration necessary. A sufficiently smart CPAN client can perform the appropriate configuration without running code from the distribution itself.

You can't avoid that in all cases; distributions with XS components, for example, need to probe system information. Good luck writing a sufficiently smart CPAN client and getting the community to agree on specific standards that let you find OpenGL headers in a cross-platform fashion, for example. Yet if 80% of CPAN distributions can get by with static, upload-time configuration, a lot of complexity of installation can go away.

Yes, that would make Module::Build and ExtUtils::MakeMaker unnecessary for (probably) most CPAN distributions, at least at the point of configuration, building, and installation. (I'm a recent fan of Dist::Zilla for automating away tedium on behalf of distribution maintainers; there's less need for Module::Install in such a world. If I never write another Build.PL again, so much the better.)

That helps, but the real problem with CPAN installations is that the CPAN itself is merely an uploading, indexing, and mirroring system. Projects such as META.yml attempt to add (and extract) meaning from the system, but they cannot work around one fundamental design feature of the CPAN. That limitation is the source of most woes for end users.

Clever readers (or experienced CPAN users) have already identified this limitation. I'll reveal it in the next installment.

David Golden praised Tatsuhiko Miyagawa's excellent new cpanminus CPAN client in The power of not being all things to all people. You should consider using cpanminus.

Don't overlook something else insightful that David wrote:

It's a lot of work to be all things to all people and I keep wondering whether making things simpler and better for 99% of people would be a better choice.

The only reliable way I've ever seen to "make the easy things easy and the hard things possible" is to make the easy things the default without preventing customization of the hard things. That's a design principle for languages, APIs, and tools.

Figure out what's most common (though not necessarily what people think they want, but what they need). Optimize for that. Consider what they might need and don't prevent it.

That's not easy, and what people need will change over time, but if you want to solve problems well, you have to solve the right problems.

I've never liked ExtUtils::MakeMaker. I've liked Module::Build from the beginning. I've never, ever liked Module::Install, even though Ingy sat in my living room and hacked on what would eventually become M::I way back several years ago.

I don't believe people who use Module::Install should be shot on sight, but I do believe that Module::Install has set the usability of the CPAN back by several years.

Ingy did identify a real problem: there's too much code strewn about the configuration and build systems of the CPAN and not enough code shared. When he found himself writing something complicated to configure, compile, and install a new distribution, he'd crib code from someone like Tim Bunce or Graham Barr or Nick Ing-Simmons. In other words, to make the CPAN—perhaps the world's largest repository of redistributable and sharable library code—work, he had to copy and paste code.

M::I did get that right; turning the configuration and build system into reusable, redistributable libraries also available from the CPAN helped reduce the amount of boilerplate code and the amount of copy and paste code in configuration systems. The people behind M::I have also helped push for better configuration of CPAN clients and better tracking of dependencies and versions and types of dependencies (optional, compilation, bundling, testing, et cetera). The CPAN ecosystem is better off for that work, even though M::I itself isn't the answer.

One of M::I's biggest failings, of course, is that it prolongs the lifespan of EU::MM. Unfortunately, I think Ingy missed the big problem when he saw copy and paste code. To do anything reasonably complex with EU::MM, you have to be able to write Perl 5 code which generates (and modifies with regular expressions) cross-platform Makefiles which themselves call into Perl 5 code because Perl 5 has a sane baseline of well-understood and reliable behavior across platforms in a way that Makefiles and shells do not.

If that's not sufficient horror, consider that the way to customize EU::MM behavior the last time I looked at it (Two notes here. First, its current maintainer refuses to add new features or change existing features because it's so awful to maintain. Second, I wrote tests for some of those behaviors, so I've read and understood the code.) you write a custom superclass called MY from which EU::MM inherits to change the behavior of the various steps of generating cross-platform Makefiles which may or may not invoke Perl 5 to perform shell functions.

I am not making this up and I did not make any typos. EU:MM inherits from your custom class.

Keeping EU::MM viable long past the point where Module::Build did everything n a saner way is but a little crime. (There's a reason almost no one recommends the use of h2xs to make skeletons for new modules anymore; most new modules aren't mere wrappers around C libraries. The need for a compilation step with a pure-Perl distribution seems more than a little bit superfluous.)

Module::Install reinvoking your current CPAN client recursively to install dependencies when your current CPAN client already has a perfectly good way to install dependencies is a slightly larger crime. (I understand the justification; what if someone is trying to install a distribution from a tarball manually without a CPAN client, but is there a Principle of Least Possible Differentiation at work with that design choice?)

I don't particularly care that using M::I as a distribution maintainer means that you have to keep track of every new release of M::I which could fix bugs or make upgrading impossible and release a new version of every one of your distributions with the new M::I because of its autobundling problems, because if you get your kicks that way, more power to you. (Don't expect me to jump up and down for joy at upgrading all of your distributions on my machines, though.)

I never particularly cared for the FUD about Module::Build from some M::I discussions, especially the nonsense about "Module::Build doesn't support --install so it spews files all over the place!" (as if EU::MM ever worked properly there) or "Module::Build doesn't support uninstalling!" (as if anyone ever used that from the CPAN client to know that it worked with any system.)

I appreciate that Module::Install provided a much nicer interface than EU::MM did, and that that interface worked transparently to hide the nasty details of EU::MM. Those are true benefits, and I don't blame anyone for choosing M::I for that reason.

Even so, Module::Install's greatest crime is that it's been a distraction from identifying and fixing the real problems of the CPAN... but that's another post.

I wrote what I understand to be the strategy behind releasing new minor releases of Perl 5. Though the development branch of Perl 5 follows a monthly release cycle, the maintenance branch currently does not. If it's difficult to predict what changes volunteer developers will make in the future, it's doubly difficult to predict which bugs they will fix (or will need to fix).

Thus any support document must explain the responsibilities of users who encounter bugs and what they should expect from developers.

A bug you discover in a new major release of Perl 5 is a candidate for a new minor release if it is:

• A security or dataloss bug
• A regression introduced in the new major release
• A failure to build on a supported platform combination
• A failing core test on a supported platform

I've organized that list in rough order of decreasing severity. The most likely candidate for a fix is the second; it indicates an undertested aspect of the system. Behavior should not change between Perl 5 releases accidentally. If a patch modified behavior on which your code depends and if that change did not occur as part of a deliberate, communicated plan, a fix is likely.

Of course, any fix in a minor release needs to maintain binary compatibility within the release family.

The easiest way (at least for developers) to find and fix such bugs is before the release of a new major version of Perl 5. That's one goal of the monthly releases: to encourage you to test all of the code you care about with versions of Perl 5 in development. That's also one reason for the Perl 5 release candidates (though it's likely too late for big fixes by then).

If you can't do that, the next step to reporting a bug is to reproduce it in the smallest example possible. 10-15 lines of Perl 5 is good. 2-3 is ideal. More than 20 is usually too big. If you can provide a test case suitable for adding to the core test suite, so much the better. From there, test your code with multiple releases of Perl 5. It helps to browse the perldelta documentation, but the amount of detail between even minor releases can be daunting. A post on PerlMonks is a good step.

If you've gone through all of that, see perldoc perlbug.

This is all no guarantee that your bug will get fixed in a minor version—you should prepare for the possibility that, given enough time since the release of the corresponding major version, the best approach for p5p is not to backport a fix to a new minor version. Even so, you will likely get one of several options:

• An explanation of why it's not a bug
• A suggested workaround
• A fix in the current version

In the latter case, you will have the option of applying the relevant patch yourself or asking someone to backport it to your own custom Perl 5 if you wish. That may not seem like the ideal situation (it isn't!), but at least with free software such as Perl, you always have that option.

When a new major version of Perl 5 comes out, history suggests a new minor release will follow.

Some of this reasoning is pragmatic. For all of the requests of the Perl 5 Porters for people to test development snapshots (5.11.0 through 5.11.5) and the inevitable release candidates (in this case, 5.12.0 RC1 through... hopefully not RC2 and RC3), nothing gets more testing or bug reports than new major releases. Bugs get reported. Changes get requested. Changes occur.

The traditional view of new major releases is that they're somewhat unstable. "Wait for the patch release," people say. This was true of 5.6.0. (The ancient Camel third edition describes an unreleased version of Perl 5 somewhere between 5.6.0 and 5.6.1). This was true of 5.8.0. This was even more true of 5.10.0, where the CPAN itself suggested that 5.10.0 was a "testing" release for bleeding edge users.

Given the size of the Perl 5 test suite and the daily and weekly test reports produced from the bleeding edge of Perl 5 itself as well as the monthly releases, most of the obvious bugs appear and get corrected quickly. Even so, bugs happen. It's software. Changes occur and people notice only in odd or complex situations. Sometimes a new compiler warning appears, or underlying libraries change. Sometimes a few updates help get Perl 5 building on a platform which itself has changed.

A patch release is inevitable.

However, the Perl 5 Porters make no promises about when a point release will occur. Nor do they promise how many point releases will occur in a family.

The Perl 5.8.0 family had nine point releases between July 2002 and December 2008. If Perl 5.10 hadn't taken five and a half years, Perl 5.8.0 might have had fewer point releases.

22 months passed between the release of Perl 5.10.0 and Perl 5.10.1. There may never be a 5.10.2.

A point release needs two things: a steady stream of bug fixed in the core without breaking source compatibility and someone to identify appropriate those patches and to make the release. In other words, one or more people need to be able to cherry-pick patches from the development track of the next release of Perl 5 to the branch which will become the new point release. The weight of history and expectations is sufficient to assume that p5p will be able to find or herd enough volunteer effort to make a Perl 5.12.1 and a Perl 5.14.1 and so on, but if monthly Perl 5 releases continue and produce a new major release every 12-18 months, the likelihood of a Perl 5.12.2 and a Perl 5.14.2 decreases.

The single unpredictable factor is the presence of a major bug discovered in that release family; a major security bug or a data loss bug is one possibility. In that case, a single-patch minor release is likely. Beyond that, minor releases have diminishing returns.