Tuesday, May 19, 2009

Shhhh... I'm having an affair!

Yep, you read it correctly- I'm cheating on my one true programming love, and I may wind up leaving her altogether. I've been a Perl monk for 3 years now, and it has been my go-to language for everything except scientific numerics and visualization. There's a good reason, too: it's easy to use, intuitive, and exceptionally powerful.

This is changing as I type, however. I've been enticed by Python, which promises to do everything that Perl can do. More importantly, it is becoming a staple in scientific programming and has powerful visualization modules available. The allure of my glue language becoming my EVERYTHING language is too much to resist. Imagine a CGI script that performs the calculation, generates the plots, and returns the requested material - and it's all in the same language! It is for reasons such as these that Python is climbing up the most used language charts.

My first task with Python is to re-create Dawkin's Weasel program, a straightforward goal (Oh, look - the good folks at New Mexico Science Watch are already on it!). I have a working version in Perl now with a CGI frontend. Let's see how quickly I can get up to speed in Python. But don't tell Perl yet; I don't think she could take it. If Python lives up to its billing, I'll let Perl down slowly.


  1. I suppose it depends on the science. And the visualizations youmay need. But as for Python climbing the TIOBE charts,
    visualizations must be the reason that Logo is climbing that same chart too.

    More seriously, there are significant flaws in the methodology TIBOE uses.
    The results it generates cannot be considered accurate for anything reflecting
    the real world. Python is a wonderful language, and learning it and as many
    other languages as you can are good things, but please don’t recommend TIOBE
    as a measure of anything useful.

  2. Thanks for the comment.
    I would certainly agree that it depends on the science; unfortunately I am miles away from biological sciences - having a research-specific Perl package would be outstanding! In space sciences, visualization comes in two main flavors: IDL and Matlab because of the complexity of the plots typically generated (spectrograms, contour plots, streamlines of flow and fields, etc.). MatPlotLib appears capable of this, but I suppose I'll find out soon enough. I've turned to Wysiwyg software on occasion, such as Tecplot, to churn out 3d plots and animations in a timely manner (using Perl wrappers to automate, of course!)

    Thanks for the links on the TIOBE index; I'll be sure to check those out. I've seen data on book sales that have supported the trends reflected in the Tiobe data, I'll try to search for a source on that. I must admit that personal experience has influenced me as well: I was disappointed to see that the O'Reilly Perl CGI book has not been updated for many years. Additionally, in my research circle Perl is viewed as an aging language with preference being given to Python.

    In any case, I am curious about what Perl can do as a visualization tool and will likely explore the modules you linked to further. Your blog will also receive a good amount of my attention, too.

    Finally, in full disclosure, I am not a true programmer in any sense of the word, but a scientist, so .... yeah. ;)

  3. While I agree that the new Perl book supply is thin, I suspect the specific reason for a lack up updates to Perl CGI is that nothing much has really changed with Perl and CGI. The community has moved on to embrace things like Catalyst to do web work and it uses FastCGI, mod_perl, or it’s own HTTP Server to serve its content. While not as easy to deploy, this performs better than CGI for larger scale applications.

    I’m really not the person to talk about visualizations, I just wanted to point out the choices that existed on CPAN. I’m also not the person to talk about astronomy related science in Perl, and while I would give my eye-teeth to be a mission specalist on a flight to the ISS (and I live in Florida!) some how I doubt they need Perl programmers who specialize in Object Orientation and XML up there. That said we do have some people, who do space-related research
    with Perl, and Perl was started why Larry was working at the JPL so there’s a history of space related Perl.

    I’m gonna try to get a few of the hard-sciences people I know who work in Perl to comment here. They may be able to help point to things better than I could.

    Finally, if you like my blog you should read the Perl Iron Man contest blogs. It’s an amazing outpouring of modern Perl usage.

  4. Now that perigrin knows, chances are Perl knows you're cheating on her... ;-)

    Anyway, I'm one of the hard scientists that he got chased out. That is, I did my master's in cosmology, but has since then worked in the commercial IT world. Nowadays, I'm working mostly on Semantic Web stuff, and I have an ambition of getting back into science in the intersection between astronomy and Semantic Web.

    Anyway, as perigrin points out, that the Perl CGI book has not been updated in a long time is not a sign that Perl is dying, it is a sign that CGI died. Actually, CGI died 10 years ago, and the Perl world has moved on. And perigrin are amongst those who knows most about where it has moved.

    I could agree that Perl 5 isn't among the most modern languages, but the Perl 6 work does promise a very modern language, and in addition, many of the thoughts that have gone into modernizing Perl also go into things like Moose.

    But then, choose the right tool for the job. I had some Matlab training in my studies and the whole Institute used IDL. Matlab was very verbose, and IDL, at least that time, was pretty bad from a CS perspective: The globally scoped variables, the complete lack of understanding of fundamental OO concepts, the common blocks you had to pass. I couldn't digest it. So, I did some looking around on my own and found R. It was originally a system for statistics, and I liked it mainly because the intuitive proximity between code and math. It has now grown really huge, and while it may not do the hefty 3D graphics that some applications require, it is great for most of your data visualization needs.

    And then, you may need some C or FORTRAN code here and there for speed.

    It is also worth noting that Greg Williams (at Rensellaer Polytechnic) is doing some of very interesting work with high-performance in-memory RDF models, RDF::Trine::Store::Hexastore, which is also an important reason (for me at least) to stay true.

    So, that's where I would be going: Use Perl for the RDF model and manipulation, for any web work and command line interaction, as well the general glue. Then use R for all computations, interfaced to the rest of the system through the RDF model using SPARQL (and possibly OWLAPI if needed).
    Where needed, interface to C or FORTRAN for optimization (someone else could write it... :-) )

    I don't think this would be very difficult to manage, even though it involves a whole bunch of programming languages, rather than just one. The RDF there at the core should take care of most of the integration needs.

  5. Hello all,

    I think many of you ask yourself, what if i had the password of my friend / girlfriend / boyfriend, associate, life partner, etc. .. to know the truth about your near partner, and reassuring that they do not hide you something.

    You have the right to be reasured !

    For all that are in need of this kind of services We come to your aid, feel free to contact us on our mail for any information, we will be happy to help you

    "Owning the information, means having the power "