2016.04.10 DCBPW, On Being Small

Welcome to the DC-Baltimore Perl Workshop!

This workshop started innocently enough. The year was 2010 -- October. We were at the Pittsburgh Perl Workshop when Brad Lhotsky said he wanted to host a similar workshop in the DC/Baltimore region. I told him no. But then he talked me and some others into it. During 2011 we worked out some details and at the 2011 Pittsburgh Perl Workshop we announced that the first DCBPW would be held April 14, 2012! We even made fliers!

Here we are six wonderful years later. We've had a few of the organizers step back over the years, mostly due to moving away, but we picked up a few others and have a core of dedicated volunteers that put this workshop together every year. We get support from The Perl Foundation and also get support from local businesses and communities. It has gone pretty well! And yet... sometimes I've felt unhappy.

You see, I started to get fantasies of grandeur.

I imagined a hundred people showing up! Two hundred! Each year this didn't happen I was a little sad. Perl dying? Community dying? What's going on that a region with probably a thousand perl developers only have 50 show up to a really awesome and cheap conference?

And then I realized -- This isn't a conference! It's a workshop.

What we have is a true "workshop". A set of people focused on learning new things, getting introduced to new ideas, and getting some hands-on time. We have organized pre-planned presentations, but we also have a lot of hallway time and variety. Then the second day is completely hands-on time. The signal-to-noise ratio is fantastic -- each attendee gets a chance to learn from their peers both formally and informally.

So yeah. It's small. That's not a bug, it's a feature :)

Add Comment

2016.03.05 Arlington RetroRuby 2016

Today I attended, a great un-conference in Arlington. I got my toehold in the local Ruby community at the Arlington meetup, and was happy to visit with lots of familiar people. I didn't meet any new people, though that was mostly because it was easy to spend time catching up.

Next time I want to jump in a bit stronger on the new-dev track, maybe walk through some hands-on exercises or do a group activity (Randori).

My favorite, besides talking to people, was the lightning talks. These have a bit more technical depth and include some code getting projected, which I love. I gave a quick this-thing-exists talk on

In the hallway track I also got to preach a bit about Module Level Polyglot :)

Add Comment

2016.01.08 Perl 6 is Separate from All, Replacement for None

In response to the "Perl 5 and Perl 6 are mortal enemies" article.

We are definitely framing this whole thing wrong. We have two choices:

  • Replacement: Perl 6 is, eventually, the replacement for Perl 5
  • Separate: Perl 6 is an unfortunately named but completely separate language from Perl 5

Most people outside of Perl 5 and Perl 6 go with Replacement just by looking at the names. Various insiders switch at random between Replacement and Separate, and I think that is the core mistake. This article has a very important sentence:

"First, a postulate: given the language similarities, the people that will find it easiest to learn Perl 6 are today's Perl 5 developers."

In reality there are SO MANY differences between Perl 5 and Perl 6, it is NOT a good postulate!

This is it -- this is the mistake where we are getting both messages at once, and the rest of the article paints a dark picture of a world (unfortunately most likely the one we live in) in which crossing these messages happens. The mixed message is a net negative for both languages.

I suggest that if you take a random [Perl 5, Ruby, Python, PHP, Javascript, R] programmer, the Perl 5 programmer might actually have MORE trouble than others due to misleading overlap and jarring differences. A developer that uses a lot of Moose might have less issues... or again might end up with the differences shocking rather than enlightening. I actually think the Ruby developer will have the LEAST trouble learning Perl 6, building on tons of overlap (everything is an object!), a shared history (sure, /foo/ is a regex, drop it in. Couple sigils here and there never hurt anybody), and similar culture (TIMTOWTDI is a motto for both). But the Ruby developer won't expect things to be mostly the same between Perl 6 and Ruby, unlike a naive Perl 5 developer.

I recommend you each shape your language to do two things. First stick strongly with Separate, second focus on the Polyglot nature and audience:

Perl 6 is a new and interesting language. It happened to be created by the creator of Perl 5 and an overlapping community, and has an unfortunate name. It is a Practical Research Language, aiming to be both cutting edge and useful. It is of interest to all dynamic-language programmers, and is especially interesting for the polyglot: you'll see familiar bits from Haskell, Perl 5, Ruby, Python, CLOS, ... plus lots of original ideas. Things that you can use, or at the very least learn and take back to your work. You can use Perl 6 now to solve small tasks, and join at the ground floor to build up the ecosystem and culture towards solving larger and larger problems.

One of the cultural things that I think can set Perl 6 apart is in being Module Level Polyglot. From Perl 5 or Ruby you can invoke Python code (both have libraries to support this that I've used) -- but you'd be hard pressed to get that put into production at your work. But NOBODY should ever implement pyplot again! If we can make doing "use matplotlib::pyplot:from<Python>" a socially-acceptable thing in Perl 6, we might be able to convince other communities to do the same... and do something amazing: revolutionize sharing cross-language modules.

Perl 6 isn't the research language for Perl 5. It is a research language for ALL languages to learn from. Perl 6 is the Borg of Languages, pulling in concepts and features to create a glorious monster. Maybe the first assimilated was Perl 5, but it clearly didn't stop there. We need to let go of its roots. Embrace that Perl 6 is Separate from All, Replacement for None.

Add Comment

2015.10.18 My First Useful Perl6 Grammar

Tags: Perl6

In my recent talks about perl6, one thing that I keep neglecting is Grammars. Part of that is because I hadn't used them yet!

I was putting together a talk on some clojure code, and was getting mad at my slide software not syntax-highlighting with magical rainbow parenthesis. Clojure being a LISP-like, it lacks some visual clues for structure and is WAY easier to glance at (in my opinion) when parenthesis are highlighted. I have vim doing this for me... but no dice in my slides.

So! Slice up slides, run them through vim, stitch them back together, and presto!

Complete code is at

I use Pinpoint as my go-to slide software. Simple and easy both to develop a deck and to present it. Each slide is a header and a body done in some Pango markup (more or less simplified HTML). Here is a 3 slide deck:

#!/usr/bin/env pinpoint

# Defaults for all slides
[font=Sans 90]


Some other slide content.

-- [font=monospace 99] [code=sh]


\# A lovely cgi application

echo "Content-type: text/html"
echo "Hello, world! Query:"

-- [font=monospace 99] [code=python]

\# Python WSGI

def application(environ, start_response):
    start_response('200 OK', [('Content-Type', 'text/plain')])
    yield 'Hello, world!'

We'll do this in two parts.

Part 1: The Grammar

#!/usr/bin/env perl6

grammar Pinpoint::Grammar {
  token TOP {

  token header { .*? <?before ^^ "--" > }

  token slide {

  token slide-header { ^^ "--" (\s* <slide-setting>)* $$ \n }

  token slide-setting { '[' .+? ']' }

  token slide-content { .*? <?before ^^ "--" > || .* $ }

Using 'grammar' is kinda like some funky cross between a regex and a class. That is -- everything is declared like a named regex, but the result is effectively a class that has a ".parse" method we can use. We're giving the whole grammar a name, and then declaring the file structure in the body.

'TOP' is the start of the parse, which will be an entire file for me. There is one special slide that is the header-slide, and then after that some content slides. Though each bit has a name, you do a lot of things just like other regexes -- so "<slide>+" means one-or-more slides.

After that it just gets broken down further and further. Probably the trickiest bit is the

<?before ^^ "--" >
pattern, which we see a few times. This says that if we see a "--" on the start of it's own line, then we've gone too far. Slide content has the "||" to say if we NEVER find the "--" then we should instead continue to the end of the file.

Part 2: The Processor

Now that we have a handy dandy grammar, we'll just slurp in whatever file/stdin we are given, parse, and the traverse the parse tree. There is a way to do this with Action objects (another class with methods for each of these tokens), but I didn't do that.

my $match = Pinpoint::Grammar.parse(slurp());

print $match<header>;

for $match<slide>.flat -> $slide {
  my $h = ~$slide<slide-header>;
  $h ~~ s/\s*\[code\=(.*?)\]//;
  print $h;

  if ~$slide<slide-header> ~~ /\[code\=(.*?)\]/ {
    my $cmd = "code-htmlize $0";
    my $filter = shell $cmd, :in, :out;
    say $filter.out.slurp-rest;
  } else {
    print $slide<slide-content>;

To make this easy, I print out the bits of the parse tree as I go. the '$h' is a chopped up version of a slide-header, removing my new and magical [code=foo] slide setting. Now that I'm looking at this, I see that I didn't actually traverse into the slide-setting list for each slide... I guess that would be better than the search/replace I'm doing here.

A specific thing to note is that "~" is forcing string-context here, so "~$slide<slide-header>" is taking the object for the slide-header node we are on and dumps it out as a string of the content. This makes sense, though I initially found the "~" usage a bit ugly.

Once we get down to a slide that has a "[code=foo]" style setting in the header, instead of printing the slide content like we do otherwise we'll run it through an external shell command. The "code-htmlize" command is some other script I wrote that takes in code and a language and outputs an HTML-colorized version using vim. Here we do the equivalent of Open2, getting a filehandle for the input and output of the external command, printing to it's input, reading from its output.

That's it! A pretty readable thing that parses and traverses my slide software file format, filtering the results as it goes.

Add Comment

2015.07.26 Exploring Clojure in the REPL

Tags: Clojure, REPL

When I first learned Linux, my friend taught me a few commands. How to to list files, how to change directories and see your current directory, how to run things, how to ask a program what parameters it takes, and how to look at a file (ls, cd, pwd, ./foo or /usr/bin/foo or foo, foo --help, cat foo or more foo). After that I just... ran a lot of stuff. I eventually discovered 'man' -- but going through /bin and running ALL the commands was really educational.

Let's get some basic tools to do that with the Clojure REPL! The major things we want to do are find things (mostly functions), and then learn all about them individually.

Finding Things

So Clojure has namespaces, which are kinda like directories, using "." as a way to indicate nesting. Vars (and thereby named functions) are kinda like files. So the full "path" to a var looks like "clojure.string/split" where "clojure.string" is the namespace and "split" is the function.

First let's get a list of all the loaded namespaces. The "ns-all" gives us this, but we want a nice sorted printout, so we'll add a bit of fancy.

user=> (doseq [n (sort (map ns-name (all-ns)))] (println n))
; ... trimmed ...
; ... trimmed ...
; ... trimmed ...

Use "dir" to look at all the exposed vars from a namespace

user=> (dir clojure.string)

Search for functions with "split" in their name. There is also "apropos" but it doesn't show namespaces -- "apropos-better" does, and "find-name" is an alias of that, so is better.

user=> (find-name "split")
(split-at split-with clojure.string/split clojure.string/split-lines net.cgrand.parsley.grammar/split-empty-prods)

You can also use a regex! Not sure how useful this is... but here we are getting all the functions that start with an "s" and in with a "t".

user=> (find-name #"^s.*t$")
(set short sort sorted-set spit split-at struct clojure.set/select clojure.string/split clojure.test/set-test net.cgrand.sjacket/shift net.cgrand.sjacket/shift-right net.cgrand.sjacket/str-pt net.cgrand.sjacket/subedit)

Learning About Things

Once you've found your function, you'll want to learn more about it and maybe give it a try.

"doc" can be used to get the documentation associated with a function. This is pretty cool! It shows the full name of the function, its signature, and it's documentation.

user=> (doc print)
([& more])
  Prints the object(s) to the output stream that is the current value
  of *out*.  print and println produce output for human consumption.

Note that you can tab-complete

user=> (doc print<tab>
print          print-ctor     print-dup      print-method   print-simple   print-str
printf         println        println-str

If you don't know what you're looking for, you can also try find-doc. This will search both the name and the documentation itself for your string or regex.

user=> (find-doc "split")
; ..... too long to include ;)

THE ULTIMATE: Get the source for a function!

user=> (source print)
(defn print
  "Prints the object(s) to the output stream that is the current value
  of *out*.  print and println produce output for human consumption."
  {:added "1.0"
   :static true}
  [& more]
    (binding [*print-readably* nil]
      (apply pr more)))

Not everything is a function. Heck, sometimes you might not know what sort of thing you're looking at. But we can find out.

user=> (type 5)
user=> (type 5.2)
user=> (type "hello")
user=> (type 'hello)
user=> (type {:a 5, :b 6})
user=> (type [1 2 3])
user=> (type type)
user=> (type dir)
CompilerException java.lang.RuntimeException: Can't take value of a macro: #'clojure.repl/dir, compiling:(/tmp/form-init5987247545872948247.clj:1:1) 
user=> (type clojure.string/split)
user=> (defn say-hi [] (println "hi!"))
user=> (type say-hi)

Getting GUI

Clojure comes with clojure.inspector, which gives some Swing GUI for exporing data structures.

user=> (use 'clojure.inspector)
user=> (inspect-tree {:a 1 :b 2 :c [1 2 3 {:d 4 :e 5 :f [6 7 8]}]})

NOTE: I had to do this to get jdk-1.7 swing apps to display while using xmonad (and other tiling window managers I guess). I now dropped this in my ~/.zshrc. Uhg.


Doing Horrible Things

So let's have some fun. First, let's get all the string function names. "dir-fn" is like dir, but returns the results instead of printing them.

user=> (def string-funcs (clojure.repl/dir-fn 'clojure.string))

Let's turn them into their actual functions.

user=> (def real-string-funcs (map #(resolve (symbol (str "clojure.string/" %))) string-funcs))

Given a function, here is how to tell how many parameters it takes (well... it might take more or less than this, but this will tell us how many params for the first definition. Good enough).

user=> (defn arg-count [f] (count (first (:arglists (meta f)))))

Here is a handy thing that tells us if the given function can deal with a single arg.

user=> (defn single-arg? [f] (= 1 (arg-count f)))

Now let's filter our string functions down to the ones that just take one argument.

user=> (def one-arg-funcs (filter single-arg? real-string-funcs))

So. We if have a one-argument function in the clojure.string namespace, let's assume it takes a string and pass in a string to it :) . The "pr-str" bit is so that we can pretty-print this with escaped newlines and such later.

user=> (defn one-arg-result [f] (pr-str (apply f ["Hello\nworld\n"])))

Now let's try this on all the one-argument string functions, printing out the function name and the result pretty like.

user=> (map #(println (:name (meta %))  "\"Hello\\nworld\\n\" ->" (one-arg-result %)) one-arg-funcs)
blank? "Hello\nworld\n" -> false
capitalize "Hello\nworld\n" -> "Hello\nworld\n"
join "Hello\nworld\n" -> "Hello\nworld\n"
lower-case "Hello\nworld\n" -> "hello\nworld\n"
re-quote-replacement "Hello\nworld\n" -> "Hello\nworld\n"
reverse "Hello\nworld\n" -> "\ndlrow\nolleH"
split-lines "Hello\nworld\n" -> ["Hello" "world"]
trim "Hello\nworld\n" -> "Hello\nworld"
trim-newline "Hello\nworld\n" -> "Hello\nworld"
triml "Hello\nworld\n" -> "Hello\nworld\n"
trimr "Hello\nworld\n" -> "Hello\nworld"
upper-case "Hello\nworld\n" -> "HELLO\nWORLD\n"


Add Comment


META INFORMATION: This is the technical blog and wiki of Brock Wilcox (awwaiid). Entries focus on my current projects, interests, and sometimes life events. If you'd like you can check out the list of All Entries or the RSS Feed. I also have a LiveJournal syndication feed for LJ friends.


Blog Blog RSS Feed







Follow @awwaiid





Wiki Edits Wiki RSS Feed

... more changes