Are you sure you're seeing all of the replies to your tweets -- even from people you don't follow? Do you ever find yourself missing important posts -- like those from your real life friends? Do you know which will reach a wider audience: new style retweets vs old style retweets?

This article will answer these and other twitter related questions. It will help beginner to intermediate twitter users get the most out of the service.

#1 Standard #Tweets

These appear simple. These show up in the timeline of your followers (@admirer and @vet in the example above). And they don't show up in the timeline of anyone who doesn't follow you (@effusive). If this was all there was to twitter it would be a complete waste of time.

The subtlety with this type of tweet is that your content may show up to someone that doesn't follow you but that is following a hashtag or a topic you mention. In the example above @hiro thows a hashtag onto the word aardvarks to indicate that it's special somehow. His tweet is then picked up by @spca who runs a constant search in a twitter client for the word "#aardvark".

The first time I realized how powerful this could be I had was at a conference. I was extremely bored by a terrible keynote. With nothing else to do I discovered the conference hashtag on twitter and started watching it. Suddenly I realized I wasn't alone. I was in a huge room full of people making fun of the presenter! It was like telepathy, or group consciousness or something. I've never felt alone or bored at a conference since.

Interesting statistic: As of June 2010, about 65 million tweets were posted each day, equaling about 750 tweets sent each second. (reference)

#2 Replies

Replies are more complicated. In the example above the guy on the left, @vet, replies to @hiro by prefixing his tweet with "@hiro". However, this probably doesn't work like you think. The tweet obviously shows up in @hiro's timeline, but it also shows up in @admirer's timeline because she follows both people mentioned in the tweet (@hiro and @vet). But the tweet does not show up to @effusive because, while he follows @vet, the author of the tweet, he doesn't follow @hiro, the person being mentioned.

Getting this scoping rule wrong is way too easy. For instance you can't just announce that someone did something great. If @hiro started a tweet with "@vet gave a great anti-aardvark presentation" then most of his followers (e.g. @effusive) wouldn't actually see the tweet! I've seen seasoned twitter users make this mistake.

As an interesting aside twitter can help track conversations. If you click a reply button in a twitter client or on the website twitter will keep track of which tweet you replied to and then help piece a conversation together. But if you simply start a new tweet with "@somone" without clicking reply, the above scoping rules still apply, but twitter won't help anyone reconstruct the conversation.

Finally a word on notifications: If someone mentions you (via a reply or anywhere in their tweet) and you follow them then you will get an e-mail and possibly an SMS notification. If someone that you do not follow replies to you, you will not get an email. For this reason you must either frequently check the "mentions and more" section of the website or use a column based twitter client, discussed in the final section on lists.

Interesting statistic: On average only 23% of tweets get a reply. (more)

Another one: 38% of tweets are conversational. In other words 38% of tweets start with an "@". If your percentage of replies is lower you probably aren't using twitter to its fullest. (more)

#3 Reply-All's

So what happens if you want to reply to someone, but still have it show up in the timelines of all of your followers? As long as your tweet doesn't start with an "@" symbol then you're fine. So the convention that has grown up is to prefix your tweet with a "." and then the "@someone". In the example above @effusive now sees the tweet where he didn't in a standard reply.

Interesting statistic: The highest usage rate of twitter to date occurred during the 2011 FIFA Women's World Cup Final between Japan and the United States when 7,196 tweets were published per second! (more)

#4 Old Style Retweet or "Retweet with Comment"

Prior to late 2009 if you saw a tweet you wanted to share with your followers you would retweet (RT) it by copy and pasting their tweet and prefixing it with "RT @author". The problem from a scoping perspective was that people would get duplicates. In the example above @admirer sees a duplicate from @vet when he retweets @hiro's tweet.

There are other problems with old style retweets unrelated to scoping. One problem is that you get long chains of "RT @person1 RT @person2 ...". Another problem is it was hard to follow a person but not all their retweets. The biggest problem is old style retweets take valuable characters away from your available 140, and sometimes require mangling the original.

Old style retweeting is still in use today primarily from people wish to retweet but add a comment, but also because it allows people to reach a wider audience (since new style retweets won't show up in searches or lists, more on this later).

Interesting statistic: Only 6% of tweets get retweeted (more)

#5 New Style Retweet

In November of 2009 twitter rolled out a new technique for retweeting aimed at solving the problems mentioned above. It caused a lot of confusion, but did solve the problems.

The end result of new style retweets from a scoping perspective is that sometimes people see tweets from people they don't follow. In the example above @effusive sees @hiro's tweet in his timeline even though he doesn't follow @hiro, because @vet performed a new-style retweet.

If you click the retweet button on the twitter website today it will perform a new style retweet. Some twitter clients like TweetDeck give you the option. The only way to perform an old style retweet or retweet with comment via the website is to copy and paste the message.

Interesting statistic: Less than half of tweets are posted using the web user interface. By far the most common twitter client is Tweetdeck with 8.48% of the market. (more)

#6 Direct Messages

Direct messages are simple. Prefix your tweet with "D somone" (no at symbol before the name) and they will be the only person that will see it. Just like e-mail except the recipient will get an e-mail and a text message if they've set that up on their phone. Note that neither people watching an included hashtag (e.g. @spca) nor people mentioned will see direct messages unless they are the recipient.

Interesting statistic: Six percent of all tweets are sent via SMS. (more)

#7 Lists

The last scoping topic worthy of mention is lists. This fantastic feature was added by twitter in late 2009. It allows you to organize the people you follow. For instance I have a private "Infrequent Posters" list that I try very hard to never miss a tweet from. Meanwhile I follow enough people that my main list is more like a stream that I dip into but don't get stressed out if I miss some of.

Lists work best when used in conjunction with a column based twitter clients (e.g. TweetDeck). These types of twitter clients are the only way to go if you want to become a twitter pro. If you've gotten to the point where you start missing content from important people (e.g. your real life friends), or you're missing responses to your tweets, or mentions from people you don't follow: then you probably need lists, but you absolutely need something like TweetDeck.

In regards to scoping, lists contain one subtlety that you need to be aware of. They do not support new style retweets. In the example above @vet posts a normal tweet. He then new-style retweets one of @hiro's tweets. @effusive follows @vet and has also placed him into a list. While the list helps @effusive not miss tweets from @vet, it does not end up displaying @hiro's tweet.

Note that this rule applies both to people who maintain their own lists as well as to people who follow lists maintained by other people (e.g. your company may maintain a public list of its twitter enabled employees).

Interesting Statistic: Pointless babble is the most common type of content on twitter. It represents 40.55% of all traffic. (more)

Conclusion

Twitter may appear to be fairly simple at first glance. In reality it has grown over the years to become very sophisticated. Sophisticated and incredibly powerful. But to fully tap that power you must understand the scoping rules and subtleties surrounding things like replies, retweets, and lists. Hopefully you're closer now.

1. Open regedit.exe
2. Navigate to HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\LanmanServer\Parameters
3. If the IRPStackSize value is present, double click on it. If it is not present, create it by right clicking in the Parameters window, selecting New, then DWord Value. Type IRPStackSize (it is case sensitive!)
4. Change the base to "decimal"
5. Choose a value that is larger than the existing value (or if you just created it, higher than the default value of 15.)
6. Restart your computer and try it now.
7. You may need to increase IRPStackSize a second time if it does not work. The maximum value you can enter is 50.
   

There were several requirements that I had to keep in mind when I was designing this

1. The cap tasks should use the environment configuration files that we already have.
2. Because there are a large number of tasks we should be able to run them individually, or with a single command we should be able to run the entire suite.
3. The cap script should produce well formatted, easy to read output, so that it's clear what's broken and where.

Control Support Methods

While building the validation tasks I found that I was having to do the same basic operations fairly often:

1. Issue a command on a remote host.
2. Parse the output of that command.
3. Issue more commands based on the output.

Or sometimes I just wanted to run a series of commands on one host, verifying that it was correct before moving on to the next. It's possible to make Capistrano work this way, but it's not well documented. By default, if you give Capistrano multiple commands to run on multiple hosts, the it will run the commands in host order. However what I really wanted is for Capistrano to run the commands in command order.

For instance: given that we have host-A, host-B, host-C, and I want to run commands doA, doB, and doC by default Capistrano will run:

    HostA > doA
    HostB > doA
    HostC > doA
    HostA > doB
    HostB > doB
    HostC > doB
    HostA > doC
    HostB > doC
    HostC > doC

But what I needed it to do was:

    HostA > doA
    HostA > doB
    HostA > doC
    HostB > doA
    HostB > doB
    HostB > doC
    HostC > doA
    HostC > doB
    HostC > doC

By using the roles that we already have configured for our deploy tasks with Capistrano we were able to create a few support methods that issue commands in the order that we want.

each_host

The each_host method is used to iterate through our configured hosts. The method prints out the hostname (which is important for the look of the script's output) sets the current host, then yields the block that was passed into it.

  def each_host
    roles[:web].each do |host|
      print_hostname host
      set(:current_host, host.host)
      yield host
    end
  end

run_serial

The run_serial method runs the given command, but only on the current host, yielding the output of the command and the name of the output stream (:out or :err). The ssh channel we don't have much use for.

  def run_serial(command)
    run command, :hosts => fetch(:current_host) do |chan, stream, data|
      yield stream, data
    end
  end

run_primary

We also have a primary server set in our configuration. We can create a few more support methods that take advantage of the primary host, for issuing command on one host only.

  def run_primary(command)
    run command, :hosts => fetch(:primary) do |chan, stream, data|
      set_current_host channel
      yield stream, data
    end
  end

currently_primary?

Having a method that tells us if we're currently on the primary host is helpful as well.

  def currently_primary?
    fetch(:current_host) == fetch(:primary).host
  end

set_current_host

The current host can also be set from the ssh channel. This method is called from the run_primary method, but this needs to be called when we call the default run method.

  def set_current_host(channel)
    host = channel.properties[:host]
    print_hostname host
    set(:current_host, host)
  end

Now we can write tasks that execute in in command order, execute tasks just on the primary host, and as we're preforming checks we print out the current host, so we're able to keep track of where we are.

Printing Support Methods

Because printing the script output out to the command line is an important part of the script I'm going to go over the printing support next. When running the server checks I wanted the output to look like rspec or cucumber. As each check is performed I wanted to see a colorful pass or fail message with the values that I was checking against to be highlighted. Also once the checks were finished I wanted to see a list of everything that failed, grouped by host.

print_hostname

This is the print_hostname method that was mentioned above. Everytime we move to a different host this method prints out the hostname.

  def print_hostname(name)
    puts "    Host: #{grey(name)} >"
  end

step

Also at the beginning of every task I wanted to print the task name and the number of the task.

  def step(name)
    step = fetch(:step)
    puts "\n[#{step}] #{name}"
    set(:step, step+1)
  end

failure

The failure method is called when a check fails. It prints the failure message, and stores it under the current hostname so that it can be printed again after all the checks have been done. If the expected or actual parameters are included it prints those along with the message, otherwise just the message is printed.

  def failure(message, expected=nil, actual=nil)
    host = fetch(:current_host)
    complete = (expected || actual) ?
      "#{message} Expected: #{grey(expected)} Actual: #{grey(actual)}" :
      message
    errors = fetch(:errors)
    unless errors[host]
      errors[host] = []
    end
    errors[host] << message
    log_item(complete, :fail)
  end

pass

The pass method is called when a check passes.

  def pass(message)
    log_item(message, :pass)
  end

log_item

The log_item method is called by the pass and failure methods. The method includes a warn and empty status which can also be used by the tasks when printing their status.

  def log_item(message, flag=nil)
    status = case flag
      when :pass : "      [#{green("PASS")}] "
      when :fail : "      [#{red("FAIL")}] "
      when :warn : "      [#{orange("WARN")}] "
      else "      "
    end
    puts "#{status}#{message}"
  end

colorize

The methods below handle the coloring of the text as it's printed to the command line. This was actually pretty fun figuring out. Every script should have fancy colorized output. (and now every script of mine will, muaha ha ha)

  def red(text)
    colorize(text, 31)
  end

  def orange(text)
    colorize(text, 33)
  end

  def green(text)
    colorize(text, 32)
  end

  def grey(text)
    colorize(text, 37)
  end

  def colorize(text, color_code)
    "\e[#{color_code}m#{text}\e[0m"
  end

Start and Finish Tasks

Now that all that's taken care of we can start writing some actual Capistrano tasks to take advantage of our fine grained control and fancy printing.

  TASK_LIST = [
    "check:all",
    "check:environment_variables",
    "check:middleware_versions",
    "check:apache_configuration"]

  on :start, 'check:setup', :only => TASK_LIST
  on :finish, 'check:print_errors', :only => TASK_LIST

setup

The setup task handles whatever specific setup needs to be done, but at the very least the step and errors variables need to be defined. They're used by the printing methods. Also, all of the tasks below are inside of the check namespace.

  namespace :check do

    task :setup do
      set :step, 1
      set :errors, {}
    end

print_errors

The print errors method is run after all the checks have been run to print a convenient list of errors grouped by host. This keeps errors from being lost in the output.

  task :print_errors do
    errors = fetch(:errors)
    if (errors.size() > 0)
      print "\n ==== #{red 'Errors'} ===="
      errors.each do |host,list|
        print_hostname host
        list.each {|e| puts "      #{e}"}
      end
    end
  end

Validation Tasks

The meat of the script are in the validation tasks. Out current script has fifteen different tasks that do everything from checking environment variables, to verifying directory and file permissions, to checking network and database statuses. If it can be automated, we'll find a way to get it in. Rather then include concrete examples though I'm just going to include some skeleton methods to show the method structure to illustrate how the support methods are used together in the Capistrano tasks.

Simple Tasks

This task executes one command on each server and validates the output from a list of expected results. This form is used by our script to check environment variables, commands on the sudo list, and programs in the cron, all of which can be read and verified with one command. I'm just including the validateVariable, and getVariable methods here to show the pass and failure methods.

  VARIABLES = [
    { :key => 'KEY', :value => /Expected Value/ },
    { :key => 'KEY', :value => /Expected Value/ },
    { :key => 'KEY', :value => /Expected Value/ }]

  task :simple do
    step "A Simple Check"
    run "env" do |channel, stream, data|
      set_current_host channel
      VARIABLES.each do |map|
        validateVariable(data, map)
      end
    end
  end

  def validateVariable(data, map)
    value = getVariable(data, map[:key])
    (value.match map[:value]) ?
      pass("Env #{grey(map[:key])} is set to #{grey(value)}") :
      failure("Env #{grey(map[:key])} is incorrect", map[:value].inspect, value)
  end

  def getVariable(data, key)
    result = data.match /^#{key}=(.*)/
    unless result
      failure "Env #{grey(key)} is not set."
      return nil
    end
    result[1]
  end

Tasks that run multiple commands on the same host

Most of the tasks in our validation script run multiple commands on the same host, using the each_host and run_serial methods. This allows us to read files and act on the values in those files. It's alse used because the script output looks better organized when running tasks in host order. The task below itterates through a list of standard directories, then a list of configured directories. For each directory a run_serial command is executed to see if the directory exists, then once the lists have been gone through it's done again on the next host.

  task :serial_example do
    step "Run multiple commands on a host"
    each_host do
      DIRECTORIES.each do |dir|
        verify_directory dir
      end

      [:deploy_to, :transfer_path, :cache_path].each do |key|
        dir = fetch(key)
        if (dir)
          verify_directory dir
        else
          falure "No directory configured for #{grey(key)}"
        end
      end

    end
  end

  def verify_directory(path)
    command = "[ -d #{path}] && echo 'true' || echo 'false'"
    run_serial command do |stream, data|
      if (data.strip == 'true')
        pass "Verified that #{grey(path)} exists."
      else
        failure "Directory at #{grey(path)} does not exist."
      end
    end
  end

Tasks using different server combinations

Here's a command that runs on the primary host, reads a file, then acts across multiple hosts. We do something like this to verify that the Apache configuration is correct. We know the httpd.conf is the same across hosts, but we want to verify that the files and directories that are in the configuration are actually on the host in question.

  task :primary_example do
    step "Verifying Certificates"
    path = fetch(:path_to_httpd_conf)
      certs = {}
      run_primary "grep SSLC[eA] #{path}" do |stream,data|

        if (stream == :err)
          failure "No http configuration file at #{grey(path)}"
          break
        end

        keys = [
          'SSLCertificateFile',
          'SSLCertificateKeyFile',
          'SSLCACertificatePath',
          'SSLCARevocationPath']

        keys.each do |key|
          certs[key] = read_http_conf_value(data,key)
        end

        pass "Read Certificate Paths from HTTP Configuration"
      end

      each_host do
        verify_file certs['SSLCertificateFile']
        verify_file certs['SSLCertificateKeyFile']
        verify_directory certs['SSLCACertificatePath']
        verify_directory certs['SSLCARevocationPath']
      end
    end

And finally we need a task that runs the whole suite of validation tasks. The all task does nothing itself, it just runs all of the other tasks.

  desc "Run all of the validation tasks"
  after "check:all",
    "check:environment_variables",
    "check:middleware_versions",
    "check:apache_configuration"

  task :all do
  end

Then to execute the cap task you would type is:

cap -q production check:all

And watch as all the beautiful passes and fails scroll by, though hopefully more of the former.

A Solution: One solution that I found was based on the template method from the GoF book. The intent of this solution was to simplify the creation of the known views of the document. The key idea of the solution is to put as much of the logic involved in document creation into to the base class.

Implementation:

abstract class Document
{
	IDictionary FormTextFields
	{
		get
		{
			return new Dictionary
			{
				{ "NameField", this.NameField },
				{ "AgeField", this.AgeField }
			};
		}
	}
	
	private string NameField
	{
		get
		{
			if(this.DocumentType == DocumentType.Big)
			{
				return "Big name";
			}
			else if(this.DocumentType == DocumentType.Small)
			{
				return "Small name";
			}
			
			return string.Empty;
		}
	}
	
	private string AgeField
	{
		get
		{
			if(this.DocumentType == DocumentType.Big)
			{
				return "42";
			}
			
			return string.Empty;
		}
	}
	
	protected abstract DocumentType DocumentType { get; }
}

class BigDocument : Document
{
	protected override DocumentType DocumentType
	{
		get
		{
			return DocumentType.Big;
		}
	}
}

Benefits: As previously stated, the main purpose of the base class is to simplify the creation of the subclasses. This solution does just that by reducing the amount of work needed to create a new subclass. The subclasses have only two main actions required for creation. The first action is to define the type of the document which is an abstract method in the base class. The second action, not shown in the subclass, is the validation of the specific concrete document type. Just doing these two actions, as oppose to adding new creation logic, speeds up the creation of new subclasses.

Two other benefits to this solution are centralization of the logic for document creation and having reduced unit testing needs for the subclasses. When the logic of the document creation is centralized into a single base class, reasoning about the document creation becomes easier. Problems arising from document creation are easier to track down. The document base class becomes the focal point for the majority of the unit tests written for document creation. The focal point reduces the need to test the subclasses since the vast majority of the behavior is in the base.

Drawbacks: A large drawback of placing all the logic relating to the document creation in the base is that the rules for creation have to be completely available. If the rules of the creation of the document are not known then the base class will be incomplete. When the base class is incomplete, it has to be reopened every time a new rule is found. The problem of adding rules leads to the main drawback of the solution: extensibility. This solution is in no way extensible since all the logic is congregated in the base class. This drawback does pose problems, however with a little rejiggering the impact of this drawback can be minimized. One possible fix, that will reduce the extensibility problem, but will increase the work to create a subclass, is to add an abstract method to the base that filters the dictionary before it is sent out. 

abstract class Document
{
	IDictionary FormTextFields
	{
		get
		{
			return new Dictionary
			{
				{ "NameField", this.NameField },
				{ "AgeField", this.AgeField }
			};
		}
	}
	
	protected abstract IDictionary FilterTextFields(IDictionary textFields);
}

class SmallDocument : Document
{
	protected override IDictionare FilterTextFields(IDictionary textFields)
	{
		textFields["NameField"] = SmallLogic();
		return textFields;
	}
	
	protected override DocumentType DocumentType
	{
		get
		{
			return DocumentType.Small;
		}
	}
}

Background

public class SortingBenchmarks extends SimpleBenchmark {
	
	private static final int SIZE = 100000;
	private static final int MAX_VALUE = 80000;
	private int[] values;

	@Override
	protected void setUp() throws Exception {
		values = new int[SIZE];
		Random generator = new Random();
		for (int i = 0; i < values.length; i++) {
			values[i] = generator.nextInt(MAX_VALUE);
		}
	}

	public void timeHeapSort(int reps) {
		for (int i = 0; i < reps; i++) {
			HeapSort.sort(values);
		}
	}

	public void timeMergeSort(int reps) {
		for (int i = 0; i < reps; i++) {
			MergeSort.sort(values);
		}
	}
	
	public void timeQuickSort(int reps) {
		for (int i = 0; i < reps; i++) {
			QuickSort.sort(values);
		}
	}

	public void timeArraysSort(int reps) {
		for (int i = 0; i < reps; i++) {
			Arrays.sort(values);
		}
	}
}

Getting Started

1. Extend the class SimpleBenchmark
2. Do any test setup/clean up in respective setUp or tearDown methods (Similar to JUnit setUp/tearDown)
3. Write the methods that will execute the code to benchmark starting with the word "time" (Again similar to JUnit, just "time" instead of "test")
4. Place the code you want to benchmark inside your timeSomeOperation methods

Getting and Running Caliper

1. Go here to get a read-only svn link to the source code
2. Check out the code then cd into <CALIPER_INSTALL_DIR> and run ant (obviously you need ant installed and on your path)

1. In <CALIPER_INSTALL_DIR>/build/caliper-0.0/lib/ there are two jar files, allocation.jar and caliper-0.0.jar. These will need to be on the classpath of your project. I chose to create a user library in Eclipse
2. JUnit will also need to be on the project classpath. Again for me it's referenced in a user library
3. I created a driver class that simply executes the main method of the com.google.caliper.Runner class. I pass in the name of my benchmark class by setting up a run configuration in Eclipse for my driver class

package bbejeck.caliper;

public class CaliperRunner {

    public static void main(String[] args) {
	com.google.caliper.Runner.main(args[0]);
    }

}

0% Scenario{vm=java, trial=0, benchmark=HeapSort} 10889428.57 ns; ?=69810.62 ns @ 3 trials
25% Scenario{vm=java, trial=0, benchmark=MergeSort} 9066618.18 ns; ?=44341.70 ns @ 3 trials
50% Scenario{vm=java, trial=0, benchmark=QuickSort} 3312312.93 ns; ?=21028.91 ns @ 3 trials
75% Scenario{vm=java, trial=0, benchmark=ArraysSort} 3104668.79 ns; ?=23965.54 ns @ 3 trials

 benchmark    ms logarithmic runtime
  HeapSort 10.89 ==============================
 MergeSort  9.07 =========================
 QuickSort  3.31 ==
ArraysSort  3.10 =

vm: java
trial:

Conclusion

View Diffs/Changes to Pages and News

One really cool feature in this release is the ability to view historical changes of a page or news item. You'll now see a "View change..." link just below the author's name on any page or news item. Tapping that link will bring up the change view shown here.

Swiping left to right will take you to older versions of the content while swiping right to left will return you to newer versions. The view for each version shows what changed from the previous version. Additionally, tapping the View button at the top right will show you the rendered content of the specific version for which you're viewing the changes. It's pretty cool!

Please see the Mini Confluence 1.1 Release blog post for full details on everything that changed in the new version.

I was actually a little bit intimidated when we arrived and started setting up our booth yesterday -- our co-exhibitors rolled into the conference with crates taller than me full of degaussers (I had to google degausser), security doors, liquid scanners, extra large safes, alarm systems, and miscellaneous oversized, super-heavy equipment looking like it was taken from the set of a 1970s spy thriller.

The truth is, we don't have 2,000 pounds of awesomeness at our booth. Not even if you throw in our pop-up display and our booth babes. Our product is software, so our coolest item is a 30" Apple monitor that we borrowed from our CEO's desk on which we display the software. But we're holding our own. We have a product that people want. One of our first visitors took one look at Clearance Track and said to my co-worker, "you're my new best friend."

We thought we would come here and meet lots of security officers from small consulting companies like Near Infinity whose security officers were overwhelmed with processing clearances. But in talking to the attendees, we're finding more government folks charged with managing the clearance data for thousands of military or other government personnel. They use JPAS, but find it insufficient to report on and manage the details of their employees' clearances. They're getting by now with Excel spreadsheets, outdated Access databases, filing cabinets, and whatever info they can remember in their heads.

Clearance Track is a great solution for them. It provides a single, organized place to store all of their clearance data. It makes it easy to report on their secure personnel, easy to find documents related to that person's clearance, easy to pass off responsibilities from one security officer to another, easy to remember to renew a badge, easy to run tedious government reports, easy to auto-generate documents... Who doesn't need easy?

So far, the reaction has been great. Lots of people are raving about how helpful Clearance Track would be in their jobs, how user friendly it is, and how much time it would save them. With a day and a half left to go, we're not missing the heavy equipment too much.

And now, if you'll excuse me, I'd better get back to our booth visitors!

Clearance Track is a fast, secure way to manage your clearances. By providing a single place to track your investigations, crossovers, badge renewals, briefings, and certs, Clearance Track significantly streamlines the security officer's job and provides a collaborative environment to organize and store employee data. Use it for 4311s, 4414s, 312s, SF-86s, visit certs, annual security refresher briefings, and key government reports.

The best conferences aren't about learning at all. They're about inspiration. I finally realized this at RailsConf last week, despite having attended dozens of conferences during the past ten years. Surprisingly, I found most of the topics rather boring, most of the presenters unpolished, and most of the presentations poorly constructed and minimally rehearsed. The things I enjoyed most were the keynotes from Dave Thomas and David Heinemeier Hansson, a performance-related presentation by Aaron Patterson, an entrepreneurial story by Tom Preston-Werner, and a reflection talk by Keavy McMinn on her journey from artist to programmer. Why did I enjoy them? Because they caused me to look at things differently, forced me to think critically about my choices, and motivated me to act. Only one of these was technical in nature by the way, just one!

A good conference should inspire you to do something. No talk is long enough to teach you anything immediately useful. The best you can hope for is exposure to lots of stuff you might want to look into after the conference. Of course you could get the same level of exposure by reading the conference agenda, Googling each topic, and reading through some documentation on each homepage. What you can't get is the inspiration from a great speaker, motivational story, or engaged community.

So the next time you contemplate attending a conference, do your best to judge the inspiration potential. After all, if you're not going to do anything different after the conference, why bother going?

select * from /logs where file="*.out" and pattern="foobar" and modified < 2 days ago
select * from /logs where file='*.out' and pattern='foobar' and modified between 20 and 30 minutes ago

find /logs -name '*.out' -mtime -2 | xargs grep 'foobar'
find /logs -name '*.out' -mmin +20 -mmin -30 | xargs grep 'foobar'

Disclaimer

Introduction

1. Eclipse 3.5.1
2. Java 6
3. The ANTLR IDE plugin for Eclipse. You could also use ANTLRWorks, the gui development environment for ANTLR. ANTLRWorks is an excellent tool, I just felt more comfortable to do this work in Eclipse.
4. ANTLR version 3.2
5. Mac OS X 10.6.2.

options, @header

grammar FQL;
options {
     language = Java;
}
@header {
     package bbejeck.antlr.fql;
}

@members

 @members {
  private StringBuilder findBuilder = new StringBuilder("find ");
  
  private StringBuilder filter = new StringBuilder();
  
  private void addString(String s){
    if(s!=null){
        findBuilder.append(s);
     }
  }
  
  private String buildTimeArg(String s, String snum, String sign){
       StringBuilder timeBuilder = new StringBuilder();
       int num = Integer.parseInt(snum);
       
       if(s.equals("days")){
           return timeBuilder.append(" -mtime ").append(sign).append(num).toString();
       }
       if(s.equals("hours")){
           return timeBuilder.append(" -mmin ").append(sign).append((num*60)).toString();
       }
       
       return timeBuilder.append(" -mmin ").append(sign).append(num).toString();
  }
  
  protected void mismatch(IntStream input, int ttype, BitSet follow) throws RecognitionException{
        throw new MismatchedTokenException(ttype,input);
  }
  
  public Object recoverFromMismatchedSet(IntStream input, RecognitionException e, BitSet follow) throws RecognitionException{
     throw e;
  }
  
}

@rulecatch

@rulecatch{
    catch (RecognitionException e){
            throw e;
      }
}

@lexer::header

@lexer::header {
  package bbejeck.antlr.fql;
}

Parsing Rules

evaluate returns [String query]
      :  query';' {$query = builder.toString() + filter.toString() ;}
      ;

query
       :   select_stmt where_stmt
       ;

select_stmt
      :  'select' '*' 'from' directory
      ;

directory
       : (p='.'{addString($p.text);} | (p='/'?{addString($p.text);}IDENT{addString($IDENT.text);})+ )
       ;

where_stmt
       :  ('where'  clause ('and' clause)* ) ?
       ;
clause
       : file_name
       | pattern
       | modified
       ;

  
file_name
       : 'file'  '=' STRING_LITERAL
         {addString(" -name ");addString($STRING_LITERAL.text);}
       ;

pattern
       :   'pattern'  '=' STRING_LITERAL
             { filter.append(" | xargs grep  ").append($STRING_LITERAL.text); }
       ;

modified
       :  modified_less
       |  modified_more
       |  modified_between
       ;

   
modified_less
       :   'modified'  '<'  INTEGER time_span                             
           { addString(buildTimeArg($time_span.text,$INTEGER.text,"-")); }                     
       ; 
  
modified_more                     
       :   'modified'  '>' INTEGER time_span
           { addString(buildTimeArg($time_span.text,$INTEGER.text,"+")); }
       ;

modified_between
       :   'modified' 'between' int1=INTEGER 'and' int2=INTEGER time_span
            { addString(buildTimeArg($time_span.text,$int1.text,"+")); }
            { addString(buildTimeArg($time_span.text,$int2.text,"-")); }
       ;

time_span
       :   'days'
       |   'minutes'
       |   'hours'
       ;

Lexer Rules

fragment DIGIT : '0'..'9';
fragment LETTER : 'a'..'z'|'A'..'Z' ;

STRING_LITERAL : '\''.*'\'';
INTEGER : DIGIT+ ;
IDENT : LETTER(LETTER | DIGIT)* ;
WS : (' ' | '\t' | '\n' | '\r' | '\f')+  {$channel=HIDDEN;};

Test Code

public class FQLTester {

public static void main(String[] args) throws Exception{
     BufferedReader reader = new BufferedReader(new InputStreamReader(System.in));
     String line = null;
     System.out.println("Enter your search:");
     while((line = reader.readLine())!= null){
         if(line.equalsIgnoreCase("quit")){
            System.exit(0);
         }
        CharStream charstream = new ANTLRStringStream(line);
        FQLLexer lexer = new FQLLexer(charstream);

        TokenStream tokenStream = new CommonTokenStream(lexer);
        FQLParser parser = new FQLParser(tokenStream);

        String parsed = null;
        try{
            parsed = parser.evaluate();
            System.out.println("parsed query is ["+parsed+"]");
            Process process = Runtime.getRuntime().exec(new String[]{"sh","-c",parsed});
            InputStream input = process.getInputStream();
            BufferedReader procReader = new BufferedReader(new InputStreamReader(input));
            String searchResults = null;
            while((searchResults=procReader.readLine())!=null){
                  System.out.println(searchResults);
            }
        }catch(Exception e){
               e.printStackTrace();
        }
      System.out.println("Enter your search:");
    }
}

• Scott Stanchfield's video tutorial on ANTLR
• Definitive Guide to ANTLR, Pragmatic Books

1. Go to http://www.adobe.com/go/supportportal If you have an account, log in. If you don't, you'll need to create one.
   
   
2. Under Get Support, Click on Orders and Returns
   
   
3. Choose the issue type Return / Exchange / Refund and then click the button that says Proceed to Online Form
   
   
4. Fill out the form, typing Cross Platform Swap in the subject line. Submit.
   -- In the notes field, here's what I typed: I am switching to a Mac and would like to do a cross-platform swap. I understand that my existing copy must be destroyed as soon as the new one arrives.
   
   
5. The next day, you'll get an email saying your issue is resolved and providing no useful information. Instructions for what to do next are actually in the ticket itself, which you can get back to on the support portal. Here's what you do:
   -- Call 1-800-833-6687 and follow the prompts to customer service (#2 then #4)
   -- Have your case number and credit card ready
   -- Give them your credit card info and verify all your contact information
   

export GOROOT=$HOME/go
export GOOS=darwin
export GOARCH=amd64
export GOBIN=$HOME/gobin
export PATH=$PATH:$GOBIN

. ~/.bash_profile

sudo easy_install mercurial

hg clone -r release https://go.googlecode.com/hg/ $GOROOT

cd $GOROOT/src
./all.bash

package main

import fmt "fmt"

func main() {
   fmt.Printf("hello world!\n");
}

6g hello.go
6l hello.6

./6.out

hello world!

What's the issue here? That Java is hard to work with? Perhaps more specifically, not just Java but perhaps the artificial complexity in developing "Enterprise" applications in Java? Nope. The problem is that this type of thinking epitomizes the attitude that business people and other "professionals" tend to have about software development in general, in that they believe it is or should be easy and that it is always the tools and rogue programmers that are the problem. Thus, with more and better tools, they reason, there won't be a need for skilled developers and the monkey-work of actually programming could be done by, well, monkeys.

I believe software development is one of the hardest activities humans currently do, and yes I suppose I do have some bias since I am a developer. Also contrary to what many people think, there is both art and engineering involved, and any given problem can be solved in an almost infinite variety of ways. Unlike more established disciplines that have literally been around for hundreds or thousands of years (law, medicine, accounting, architecture, certain branches of engineering like civil, etc.), the software industry hasn't even reached the century mark yet! As a result there isn't any kind of consensus whatsoever about a completely standardized "body of knowledge" and thus there isn't an industy-recognized set of standard exams and boards like you find in the medical and law professions for example. (That topic is for a future post.)

One thing that is certain is that software development involves logic, and thus people who can solve problems using logic will always be needed, whether the primary medium stays in textual format (source code) or whether it evolves into some different representation like Intentional Software is trying to do. So the statement from the article that "you needed to be a highly trained programmer to make heads or tails of the language" is always going to be true in software development. More generally, highly skilled people are needed in any complex endeavor, and attempts to dumb dumb complex things will likely not succeed in any area, not just software development. Would you trust someone to perform surgery on you so long as they have a "Dummies Guide to Surgery" book? Or someone to represent you in court who stayed at a Holiday Inn Express last night?

I hypothesize that things are becoming more complex as time moves on, not less. I also propose that unless we actually succeed in building Cylons who end up wiping us all out or enslaving us, we will never reach a point where we don't need people to actually think and use logic to solve problems. So even though many business-types would love to be able to hire a bunch of monkeys and pay them $0.01 per day to develop software, those who actually realize that highly skilled people are an asset and help their bottom line, and treat them as such, are the ones who will come out on top, because they will smash their competitors who think of software/IT purely as a cost center and not a profit center.