I started the project as an effort to protect myself and others from Firesheep when using Safari on an open public wireless network, much like those found in coffee shops, hotels, and airports everywhere. Firesheep works by hijacking your session, which is basically a way of stealing your active login without needing your username or password.

My goal with SSL Everywhere was to protect someone from a session hijacking attack by ensuring all requests to the originating website were tunneled through SSL. This seemed trivial until I thought about the various kinds of requests that would need to be secured.

• the original HTML page
• images
• external JavaScript files
• external CSS files
• inline frames (iframes)
• object or embed tags for things like videos or applets
• Ajax requests
• requests for the favicon

It wasn't until I started to tackle each of the list items that I realized just how limited Safari's extension API is, and ultimately that one could never build a foolproof extension to protect Safari users from session hijacking attacks.

Lessons Learned

If you've spent any significant time writing Safari extensions, you may already be aware of the many restrictions and challenges I wrestled with for hours. Much of what I learned was trial and error, despite well written documentation from Apple. While the documentation does a fair job of describing many of the things you can do with an extension, it doesn't provide as much detail on what you can't do. That's what you'll find below.

No Access to Raw Cookies

The key to avoiding a session hijacking attempt is making sure the attacker can't access the session cookie, or cookies, your browser sends on each request. This can obviously be done by making sure all requests take place over an SSL connection. Unfortunately, you can't guarantee this won't happen since Safari extensions have no opportunity to intercept webpage requests before they occur. A user could easily type http://twitter.com into their address bar and SSL Everywhere couldn't stop the request.

The other option I pursued was having SSL Everywhere attempt to mark session cookies as secure since the browser will not send cookies marked as secure over a non-SSL connection. However, a Safari extension has no additional ability to manipulate cookies than normal JavaScript running on a page, meaning there is no way to read a cookie's path or expiration date for example. So, there's no way for the SSL Everywhere extension to simply mark a cookie as secure without having to guess at, or omit, other cookie information which may be important to the operation of the website.

No beforeload for favicons, Ajax Calls, or Stylesheet References

Apple was responsible for adding the beforeload event to Webkit. This gives scripts (and extensions) the ability to decide whether an external resource should be loaded or not. As stated in the Safari Extensions Development Guide,

Safari 5.0 and later (and other Webkit-based browsers) generates a "beforeload" event before loading each sub-resource belonging to a webpage. The "beforeload" event is generated before loading every script, iframe, image, or style sheet specified in the webpage, for example.

I was thrilled after reading those two sentences because it was exactly what I needed. That is, until I discovered there's no beforeload event fired when the browser requests a website's favicon, makes an Ajax request, or loads an image reference defined in a stylesheet. This leaves a loophole where I can't stop a non-secure favicon reference, Ajax call, or CSS image reference from exposing the session cookies I can't properly mark as secure.

src Attribute Can Not be Changed in beforeload Event Handlers

I ultimately wanted to have a beforeload event listener that would

1. inspect the source URL being requested
2. rewrite the source URL to a secure https: version if necessary
3. replace the resource's insecure reference with the secure https: URL
4. allow the loading of the resource to proceed with the new, secure URL

Following the above procedure results in the resource being requested with the SSL-secured URL as desired, but will not stop the original load with the original, insecure, URL. You just end up making two requests for the same resource; one over SSL, the other not. Again, another point of exposure for the cookies.

beforeload != before request

It turns out that all the effort described above to leverage the beforeload event was futile. Preventing a resource from loading, as described in Apple's example of blocking unwanted content, stops it from being inserted into the DOM, but does not prevent it from being requested by the browser anyway.

Apple's example documentation states

If your script responds to a "beforeload" event by calling event.preventDefault(), the pending sub-resource is not loaded. This is a useful technique for blocking ads...

Should I have really expected that "the pending sub-resource is not loaded" doesn't imply that it's not requested? Perhaps I wasn't sharp enough to catch that nuance, but once again the cookies are exposed.

Host Page Prototypes Can Not be Changed

When I discovered the beforeload event doesn't fire on Ajax requests, I decided to try to override the XMLHttpRequest open method and rewrite any insecure URLs to a secure version before yielding to the original open implementation. What I found was that you can't modify the prototypes of the page your start or end scripts are being injected into.

I suspect this has something to do with start and end scripts being secluded in their own separate, and randomly named, namespace. Changing around object prototypes never resulted in any errors, it just didn't change the prototypes of the host page and therefore couldn't change the XMLHttpRequest object. Again, another point of exposure for the precious cookies.

Status of SSL Everywhere

If you've read this far it should be obvious that SSL Everywhere cannot guarantee protection against session hijacking attacks, including those from Firesheep. However, it does enhance security by automatically redirecting you to secure versions of many websites and rewriting insecure links to their SSL-encrypted equivalents. If you must use Safari to access popular websites when connected to an open WiFi network, you're probably better off doing it with SSL Everywhere.

We don't offer a pre-built version of the extension for easy installation into Safari because we don't want people to casually install it and forget that it's not completely secure for all sites. If people find it valuable nonetheless, then we may consider creating the extension bundle in the future.

Try It!

If you'd like to try SSL Everywhere you can find the source code on Github. It's open source software licensed under the GPL version 2 license, primarily because it borrows code from HTTPS Everywhere. If you can come up with solutions to any of the problems I encountered, please let me know!

I was actually a little bit intimidated when we arrived and started setting up our booth yesterday -- our co-exhibitors rolled into the conference with crates taller than me full of degaussers (I had to google degausser), security doors, liquid scanners, extra large safes, alarm systems, and miscellaneous oversized, super-heavy equipment looking like it was taken from the set of a 1970s spy thriller.

The truth is, we don't have 2,000 pounds of awesomeness at our booth. Not even if you throw in our pop-up display and our booth babes. Our product is software, so our coolest item is a 30" Apple monitor that we borrowed from our CEO's desk on which we display the software. But we're holding our own. We have a product that people want. One of our first visitors took one look at Clearance Track and said to my co-worker, "you're my new best friend."

We thought we would come here and meet lots of security officers from small consulting companies like Near Infinity whose security officers were overwhelmed with processing clearances. But in talking to the attendees, we're finding more government folks charged with managing the clearance data for thousands of military or other government personnel. They use JPAS, but find it insufficient to report on and manage the details of their employees' clearances. They're getting by now with Excel spreadsheets, outdated Access databases, filing cabinets, and whatever info they can remember in their heads.

Clearance Track is a great solution for them. It provides a single, organized place to store all of their clearance data. It makes it easy to report on their secure personnel, easy to find documents related to that person's clearance, easy to pass off responsibilities from one security officer to another, easy to remember to renew a badge, easy to run tedious government reports, easy to auto-generate documents... Who doesn't need easy?

So far, the reaction has been great. Lots of people are raving about how helpful Clearance Track would be in their jobs, how user friendly it is, and how much time it would save them. With a day and a half left to go, we're not missing the heavy equipment too much.

And now, if you'll excuse me, I'd better get back to our booth visitors!

Clearance Track is a fast, secure way to manage your clearances. By providing a single place to track your investigations, crossovers, badge renewals, briefings, and certs, Clearance Track significantly streamlines the security officer's job and provides a collaborative environment to organize and store employee data. Use it for 4311s, 4414s, 312s, SF-86s, visit certs, annual security refresher briefings, and key government reports.

Grant's model security is deliberately designed to force the developer to make conscious security decisions about what CRUD operations a user should be allowed to perform on your model objects. It doesn't care how you choose to authenticate and authorize your users to perform a CRUD operation, it only cares that you actually do it.

Rather than specify which operations are restricted, Grant restricts all CRUD operations unless they're explicitly granted to the user. It also restricts adding or removing items from has_many and has_and_belongs_to_many associations. Only allowing operations explicitly granted forces you to make conscious security decisions. While it obviously can't ensure you make the correct decisions, it should help ease the latent fear that you've inadvertently forgotten to secure something.

Enough talk, let me show you an example of how you might use it. To enable model security you simply include the Grant::ModelSecurity module in your model class. In this example you see three grant statements. The first grants find (aka read) permission to everyone. The second example grants create, update, and destroy permission when the passed block evaluates to true, which in this case happens when the model is editable by the current user. You can put any code you want in that block as long as it returns a boolean value. Similarly, the third grant statement permits additions and removals from the tags association when it's block evaluates to true. A Grant::ModelSecurityError is raised if any grant block evaluates to false or nil.

class EditablePage < ActiveRecord::Base
  include Grant::ModelSecurity
  has_many :tags

  grant(:find) { true }
  grant(:create, :update, :destroy) do |user, model| 
    model.editable_by_user? user 
  end
  grant(:add => :tags, :remove => :tags) do |user, model, associated_model| 
    model.editable_by_user? user 
  end

  def editable_by_user? user
    user.administrator?
  end
end

There's a lot more to the grant statement than shown in the above example. For instance, you can have multiple grant statements for the same action. Ultimate permission to perform the action will not be granted unless all grant blocks evaluate to true.

As you can see, Grant is pretty simple to use, but it's not going to do the dirty work for you. It's up to you to make the proper security decisions. Grant's just there to make sure you don't forget.

public class TestIndexerSearcher {

   public static void main(String[] args) throws Exception {
      Directory directory = new RAMDirectory();
      IndexWriter indexWriter = new IndexWriter(directory, new StandardAnalyzer());
      indexWriter.addDocument(buildDocument("DATA:sametoken","ACL_FIELD:access"));
      indexWriter.addDocument(buildDocument("DATA:sametoken","ACL_FIELD:noaccess"));
      indexWriter.optimize();
      indexWriter.close();

      IndexSearcher indexSearcher = new IndexSearcher(directory);

      QueryParser parser = new QueryParser("DATA", new StandardAnalyzer());
      Query query = parser.parse("sametoken");
		
      //This is all you have to add to your existing code.
      Filter aclFilter = applyAccessControl(new TermQuery(
         new Term("ACL_FIELD","access")));

      Hits hits = indexSearcher.search(query, aclFilter);
      System.out.println("Hits[" + hits.length() + "]");
      for (int i = 0; i < hits.length(); i++) {
         Document doc = hits.doc(i);
         System.out.println("DATA [" + doc.get("DATA") + 
            "] ACL_FIELD [" + doc.get("ACL_FIELD") + "]");
      }
      indexSearcher.close();	
   }

   private static Filter applyAccessControl(Query aclQuery) {
      return new CachedQueryFilter(aclQuery.toString(), 
         new QueryWrapperFilter(aclQuery));
   }

   private static Document buildDocument(String... fieldInfo) {
      Document document = new Document();
      for (int i = 0; i < fieldInfo.length; i++) {
         String[] split = fieldInfo[i].split(":");
         String fieldName = split[0];
         String fieldValue = split[1];
         document.add(new Field(fieldName,fieldValue,
            Field.Store.YES,Field.Index.TOKENIZED));
      }
      return document;
   }	
}

public class CachedQueryFilter extends Filter {
   private static final long serialVersionUID = 6797293376134753695L;
   private Filter filter;
   private String key;
   private static transient Map<String, BitSetCache> filterCache = 
      new ConcurrentHashMap<String, BitSetCache>();

   public CachedQueryFilter(String key, Filter filter) {
      this.filter = filter;
      this.key = key;
   }

   public BitSet bits(IndexReader reader) throws IOException {
      BitSetCache cachedBitSet = (BitSetCache) filterCache.get(key);
      if (cachedBitSet != null) {
         BitSet bitSet = cachedBitSet.bitSet.get();
         if (bitSet != null && cachedBitSet.indexReaderVersion == reader.getVersion()) {
            return bitSet;
         }
      }
      BitSet bits = filter.bits(reader);
      BitSetCache bitSetCache = new BitSetCache();
      bitSetCache.indexReaderVersion = reader.getVersion();
      bitSetCache.bitSet = new SoftReference<BitSet>(bits);
      filterCache.put(key, bitSetCache);
      return bits;
   }
	
   private class BitSetCache {
      long indexReaderVersion;
      SoftReference<BitSet> bitSet;
   }
}

Requirements

• Fuzzy searching is a must (Soundex, Levenshtein, etc.)

• Has to be fast, a must with any searching solution

• Has to provide access control

• Full data load indexing needs to be completed in a reasonable amount of time

• Scoring needs to be a custom implementation

• Needs to run on a predetermined environment, meaning that new hardware purchases are not going to happen any time soon

• And last but not least is ability do all these things on a dataset that exceeds a billion records

So we have had a lot of constraints to deal with, the hardest one by far is the last one.

The Data

• 1 billion plus records

• Over 30 million unique terms

Indexing and Searching Server Specs

• 20 CPUs

• 32 Gig of ram

• Dedicated SAN storage

First Searching Experiences

After getting the index built in multiple partitions, I fired up a simple Lucene console to do some simple searches with a Lucene multi searcher. Ran out of memory with 2 Gig heap, tried the maximum heap size for the 32 bit JVM we were using, 3.3 Gig, and that ran out of memory as well. So, initial tries to just run one search were unsuccessful.

Then we installed a 64-bit JVM and tried an 8 Gig heap, and it worked! I could run searches and after the first couple of warm up searches it was getting 20 - 80 ms responses on single term searches. Great, but then we tried a Fuzzy search, which uses a Levenshtein algorithm to calculate matches, 2 minutes 45 seconds, this was unacceptable.

Next we wrote our own Levenshtein Lucene query and got the 2 minutes plus search down to about one second. We found that the built in Lucene Fuzzy query was taking 85-95% of the time to find the terms to search. Then after those terms were found the actual search with those expanded terms only took a second to two depending on how many terms were found. So we replaced the built in Fuzzy query with a custom one that gets near instantaneous results on Levenshtein fuzzy matches. Problem solved.

Indexing Time

After our initial proof of concept was complete, we needed to improve the indexing time down to something more reasonable. The index creation from scratch was taking 36 - 48 hours to build with 20 CPUs running at 100% utilization. Which means that the machine was indexing about 9,000 records a second. Not bad for Lucene 2.2, but not that great.

First we stopped merging the indexes after we created them, that by itself was taking about 12 hours. At this point we also started searching these multiple indexes in parallel, and we are seeing modest increases in query performance.

Second, we upgraded to Lucene 2.3, this provided a huge increase in indexing speed. Our index creation time went from 36 - 48 hours (depending on if we merged indexes or not) down to 3-4 hours. The indexing process is now indexing around 125,000+ records a second. Huge improvement, if you haven't upgraded to 2.3, you should!

Current Development

We are in the process of adding access control to Lucene as well as adding new custom queries and scoring. So far Lucene has performed better than any of the competition that it has come up against, and with it's price point it seems to have won acceptance on our project.

In upcoming parts I will go into more details about the technical solutions that we have developed to solve these problems, as well others that I haven't mentioned yet.

Internal Session Id 1207429769869209087112251146956
User Session Id H3qJ4YGjTnTH1Sv0d4nVMBNhr2vdn77m4MKGQ3MT0SVVhQQvsQBk!1447771105!1207429769869
telprdB UserIntB12 java.lang.NullPointerException

That's a lot more information than should be given out to anyone, and is certainly not "user friendly." Do you think they are using Java? That NPE didn't give it away did it? Then again you can pretty much figure that out from the ".do" on the end of the URLs they use, which is one reason why web frameworks these days allow you to map things in a more REST-friendly and technology-agnostic manner using *.html or something like /my/site/person/1. Another rule of thumb: design URLs to be technology agnostic and generic, so that just from a URL it cannot be determined what technology you are using and in case you need or want to switch to a different technology you could theoretically use the same routing scheme in your URLs, which would allow bookmarks to keep working.

HTML filtering was never recommended because it was so difficult to get right, and with no proven libraries, trying to build a solution would almost certainly contain security holes. Thanks to Arshan Dabirsiaghi we finally have something to use. He has created the OWASP AntiSamy project to easily sanitize HTML input. AntiSamy is currently implemented as a Java 1.5 compatible library, but there are plans to support other platforms.

For the record, I don't believe there is any absolutely correct answer. There are valid arguments for each side of the debate. My perspective has mostly been shaped through experience developing custom software for clients. I have little experience developing commercial software and surmise that your thoughts on the subject may differ if that's your day job.

In my experience, most customers view their data as the most important aspect of any system, not the application. They want to be sure their data is safe at all times. What if someone accesses the database through a different application, a third-party data broker or directly via a SQL client? How do you protect the data in those situations if all the restrictions are resident in the application? I hear these questions all the time.

The rebuttal to this argument that I hear most often is that nobody will ever access the data outside of the application. This seems like an obviously shortsighted statement to me and I don't think anyone actually believes it. It's more of a synonym for saying that you'd rather not worry about security at the database layer because it's out of your control, too hard, etc. Pick your favorite excuse. I've probably used them all myself.

What it all boils down to is that the database is very likely to outlive your application. In fact, it'll probably outlive your application and several subsequent ones. Why would a customer want to pay someone to rebuild the security logic each time? It's tedious, expensive, and error prone. Additionally, you're application may not be the only one accessing the database. What are the odds that two development teams implement security constraints the same exact way in each application? Not good, I promise.

As I stated in the beginning, I've definitely changed my perspectives on application versus database-level security. I was certainly an application-level security bigot before, but I find it difficult to continue my unwavering ways when looking at the problem more critically. I just hope my fellow colleagues don't renounce me for siding with the DBAs on this one.

• Cross-site scripting (XSS)
• Cross-site-request forgery (CSRF)
• JSON Hi-jacking
• JavaScript portscanning
• JavaScript and CSS History "Go Fish"

1. Hardware-Encrypted Disk Drives
2. Solid State Disk Drives

I believe the first topic is a direct result of the disclosure of scores of high-profile data breaches resulting from stolen computers. The highest profile instance occurred in May of 2006 when a U.S. Department of Veteran's Affairs laptop containing the names, social security numbers, dates of birth, phone numbers, and addresses of 26.5 million American veterans was stolen. Unauthorized information disclosure has unfortunately become an all too common occurrence in today's world.

In response, vendors such as Seagate have introduced hardware-encrypted notebook hard drives to counter the unintended loss of sensitive information when a computer is stolen. The idea is that data stored on the hard drive is encrypted at all times, rendering the information practically useless to thieves. The encryption is done in specialized hardware to reduce the performance impact associated with the extra computation.

The second topic is the result of a continued increase in flash-based memory capacity to the point where it can now be considered as an alternative to traditional hard drives. Both Samsung and Sandisk have recently announced 64GB (Samsung) and 32GB (Sandisk) products targeted for laptop usage. Since there are no moving parts, flash-based memory consumes much less power than the spinning platters of a traditional hard disk, which is obviously a big benefit for laptop users. As a side benefit, these flash-based memory "drives" are rumored to be anywhere from 4-10x faster than a traditional hard drive, which brings me to the main point of this post.

How long will it be until we see solid-state "drives" such as the ones from Samsung and Sandisk coupled with dedicated hardware-based encryption as demonstrated on the new Seagate drive? A considerable downside to full-disk encryption of traditional drives is the performance impact. Since the new flash-based drives are several times faster, why not bake in full-drive encryption now when nobody will notice it. What you'd end up with are flash-based drives that provide full-disk encryption and still outperform traditional drives. Waiting to add full-disk encryption to later versions of flash-based drives will only introduce the performance debate that faces today's traditional drives, making the purchasing decision all that much more difficult. Why not avoid the confusion and help customers better secure their data in the process?

<table>
<c:forEach var="book" items="${books}">
<tr>
<td><c:out value="${book.title}"/></td>
<td><c:out value="${book.author}"/></td>
<td><c:out value="${book.isbn}"/></td>
</tr>
</c:forEach>
</table>

with this

<table>
<c:forEach var="book" items="${books}">
<tr>
<td>${book.title}</td>
<td>${book.author}</td>
<td>${book.isbn}</td>
</tr>
</c:forEach>
</table>

As you can see, the syntax in the second example is much cleaner. But, “with great power comes great responsibility.” The chink in the armor that every tutorial I’ve read (including Sun’s Java EE 5 Tutorial fails to mention is that the expression syntax does not escape HTML characters. Therefore, any web application using JSP Expressions to output unsanitized, user-entered data will be vulnerable to Cross Site Scripting (XSS) attacks.

Unfortunately, the cure for this XSS vulnerability leads us right back to our first example. As section 2.2.2 of the JSP 2.0 Specification reads, “In cases where escaping is desired (for example, to help prevent cross-site scripting attacks), the JSTL core tag c:out can be used.”

My advice is to use JSP Expressions only as tag library attribute values and stick to using JSTL’s c:out tag for writing text to a page. Deciding which instances of template text expression usage are safe and which are not is simply too error prone and the consequences of a mistake too dangerous.