Tuesday, July 31, 2012

FlightStats Names Tech Industry Veteran Clent Richardson to Board

FlightStats is growing and has appointed Clent Richardson to the Board of Directors. The company plans to leverage Clent's vast experience in consumer, publishing and mobile technology companies to support its rapid push for growth in mobile consumer travel.
Clent Richardson

Clent Richardson is Managing Director of Huntington Hill Ventures and brings more than 25 years of C-level executive experience developing winning strategies and delivering flawless execution with companies such as Apple, AOL, AVG, Immersion, TiVo and T-Mobile where he drove organizational and operational transformations by building high performance teams and creating vision to deepen customer loyalty, drive efficiencies, focus, and deliver key business metrics.
“FlightStats is a hidden gem, powering countless real-time flight updates and airline stats for carriers and airports globally, and I look forward to working with Jeff and the board to help bring FlightStats into the forefront of direct-to-consumer usage and to help grow their global footprint.” ~ Clent Richardson
Clent joins the FlightStats Board at a time when the company is making a significant push in the consumer-focused mobile travel market while maintaining the 40% year over year growth in the business-to-business sector. FlightStats has 41 employees, up from 35 a year ago.

Monday, July 23, 2012

Wm Leler talks about moving away from Google Maps into the world of Open Source



Filmed at Open Source Bridge 2012 (http://www.opensourcebridge.org)

Wm Leler is a Principal Engineer at Flightstats (http://www.flightstats.com). In this interview, he talks about the advantages of moving away from Google Maps and what open source maps can offer in terms for flexibility and customization.

You may also want to see Wm's slides for his presentation "When Google Maps Gives you Lemons, Make Lemonade" at  Open Source Bridge 2012.


Friday, July 20, 2012

Switching Mapping APIs

Like many companies who do a lot of mapping, FlightStats had a bit of a shock in October 2011 when Google announced that they would start charging top users of the Google Maps API. We were very heavy users of Google Maps and the cost would have been a considerable amount of money for us. So we started looking at alternatives.

I don't begrudge Google for charging for their maps, but our needs were actually fairly modest. We didn't need any of the fancier features like driving directions, traffic, streetview, or 3D buildings; we mainly needed to show the location of airplanes. At the same time, we had spent a lot of time modifying and extending the Google Map API to add our own layers for things like weather and airplane routes. We had even developed our own basemap for some of our apps. But like many companies, we had started using Google Maps when it was the only game in town and had never looked around later to see if there was something else that would fit our needs.

The Top Seven Alternative to the Google Maps API
Luckily for us, there was. Not only that, but many of the alternatives were not just free, but had significant advantages for us. To help other people who are looking to replace Google Maps, we recently published an article  reviewing all the major alternatives, both proprietary and open source.





 You can read this article at:

http://www.netmagazine.com/features/top-seven-alternatives-google-maps-api

In the end, we picked the Leaflet map API, built by people at CloudMade but open source. Leaflet is a fairly new library, but it has become very popular. It had all the features we needed and is very easy to extend and customize. This made it much easier to build things like dynamic maps, where airplanes move around in real time, or with animations of weather radar. It also works well on mobile devices.

For basemaps, we had one that we had built ourselves, but we also found that there are a wealth of free maps out there. The Google basemaps are mainly designed for automobiles, but we found maps that were better for displaying our information (mostly by having less detail in the basemap itself, making it easier to see flight positions or weather overlays).

I knew we had done the right thing when Google announced that they were lowering their price by 88%. I asked around if anyone regretted the effort we spent moving to a new API and maps. The answer was no.

I really want to thank Google for inventing the technology that allowed people and companies like us to create interactive map applications. But now I'm really happy that there is some competition and variety, so we can pick the API and maps that are most suitable for what we are trying to do.

Monday, July 16, 2012

FlightStats Data Sources


FlightStats aggregates schedules, airline, airport, and positional data from over 300 sources to produce its industry leading Flight Status information services.  From these numerous sources, we handle over 20 million pieces of flight data each day.  Using our internal processes we validate, normalize and apply changes to our Flight Status database resulting in millions of flight status and positional updates.

Published Schedule Data

Published Airline Schedules is the first ingredient in building flight status.  We ingest a daily schedule file produced by Innovata and normalize the data.  Data elements include routing, scheduled block times, codeshares, equipment type, flight type (cargo / passenger), and traffic restrictions.  Using this data we create upcoming flights in our database to which we update with operational data.

Airline and Airport Data

Through direct connections, feeds and reservation systems, FlightStats receives the operational data reported by airlines on their flights.  Data elements obtained include estimated and actual block times, status, and gate assignments. We process information from numerous protocols and formats including HTTP, FTP, Web Services, and MQ Series.   

Positional Data

We rely on two information streams to obtain positional / tracking data.  We process the FAA Aircraft Situation Display to Industry (ASDI) and integrate Automatic Dependent Surveillance-Broadcast (ADS-B).  Collectively, the data from these providers allow us to track the flight plan, latitude, longitude, altitude, and speed of many of the world’s flights.   

Reference Data

We maintain reference tables of airlines, airports, timezones, and equipment.  Each incoming piece of information is checked against these tables.  Where necessary we maintain distinct mapping / translation rules for a given data source.

The end result of our efforts is a comprehensive database of Flight Status information. FlightStats consistently obtains arrival times on over 77.5% of the world’s published flights, and we’re actively pursuing relationships to increase coverage. Flight Status records exist for each flight containing the published data, current airline reported information and positional data.  We make these records available in a variety of products including APIs, Alerting Services, Data Feeds and FlightStats.com.

Tuesday, July 10, 2012

You Ask, FlightStats Answers - Announcing Improvements to the Beta APIs

Thanks to everybody who has taken the Flex APIs for a test drive and given excellent feedback and suggestions. We took your suggestions to heart and refined the APIs to better serve your needs. The services are greatly improved and we think you'll be pleased. We intend to make the modified API available on the beta site within the week.

In the meantime, we'd like to give you an overview of the upcoming changes. After the overview we'll include some example data for our fellow propeller-heads.

Your suggestions included:

  1. Reduce the response sizes
  2. Include summarized delay calculations in Flight Track
  3. Decorate responses with contextual request data
  4. Include airport geographical coordinates in Flight Track, always
  5. Return friendly and flexible error responses

1. Reduce the Response Sizes

Users making mobile and other interactive applications noticed that our default responses were rather bulky, contributing to slower applications and unnecessary bandwidth consumption. We're adding two features to help reduce response sizes.

First, we are normalizing repetitive data into a Data Appendix that contains the ancillary data exactly once, keyed by our unique FlightStats code.  All flights contain common data such as airports, airlines, and equipment. Instead of every flight repeating identical data, the flight structure will now reference the data in the appendix. To obtain the complete details (for example, the airline), you will match the FlightStats Code in the flight to the appendix entry with the same FlightStats code.

We will continue to offer the data in original repetitive "inline" format, enabled by a request parameter.



Second, we'll offer an option to filter out the Flight Status Updates portion of the flight data, which contain a change history of every update we've made to the flight data and are unnecessary in many applications. This data is frequently the largest portion of the flight data structure and removing it will reduce the response size significantly.



2. Include summarized delay calculations in Flight Track

Some of our customers asked for simple delay information in the Flight Track responses, so they could display the delay information in their maps. We think this is a clever idea, so we will include a single "delay" measurement for each flight in the Flight Track response.


The flight data in the FlightStatus response will still contain the detailed delay calculations (gate, runway, etc...).

Don't forget that all the Flight Track responses contain flightIds that can be used with the Flight Status services to look up in-depth Flight Status data.




3. Decorate responses with contextual request data

Our APIs have several optional parameters, each with a default values.  Some required parameters require interpretation on our part, such as to distinguish ICAO, IATA, and FlightStats codes. As we were responding to customer feedback, support requests, and bug reports, we realized that it would be helpful if the response contained information summarizing the request, removing ambiguity about our interpretation of the request parameters, and returning additional information about entities in the request.

We will return information about the request, including:
  • The requested URL
  • The request parameters, including default parameter values
  • How we interpreted or normalized the values, particularly codes
  • For error responses, we will include any information about parameter validation failures
As we introduce the next two changes, you'll see how we use this feature to create a more usable API.
See an example of the response with contextual request information



4. Include airport geographical coordinates in Flight Track

Users drawing maps based on the Flight Track APIs encountered a problem:  When a Flight Track request for an airport returned zero flights, they didn't receive the airport data they need to render the airport on the map (lat/lon, name, address, etc...).

Thanks to the changes we've already described, we have an elegant solution. The request data section of the response will contain the requested airport and all the information we have for it. Regardless of whether flights were returned, the user will have the airport information they need to render the airport on the map.

The airport details will be in the Data Appendix or directly in the request data structure, depending on the format style you request.
 
See an example of a Flight Track response with no flights and airport information




5. Friendly and flexible error reporting

In the initial beta release, we only returned application error responses (parameter validation failed, authentication failed, data not found, bogomip parallax error, etc...)  using traditional HTTP error codes with plain text messages. While some users were satisfied with this approach, other users found it difficult to weave the error responses into their applications. In particular, AJAX developers using our JSONP services found that error handling with non-200 HTTP codes is nearly impossible. In addition to solving these woes, we wanted the ability to return validation errors for multiple parameters in a way that JSON/XML mapping APIs would automatically parse the response into local data structures.

To satisfy everybody's needs, we will offer two error formats in our APIs, controlled using a request parameter.

First, we will continue to offer the same application error responses as in the initial beta release: the error will be reported with a 400-level HTTP code and a simple plain-text error message. Transport errors will be reported with 500-level HTTP responses. To continue receiving this error response format, you will need to provide a specific input parameter.

The new error response format will now be the default. Application errors will result in a 200 response code, but the response will contain an error structure. The published XSD will contain the error structures, so your mapping software should automatically read and parse the errors.

In both formats, we will use the previously mentioned request data to return per-parameter validation errors.

Note that these response formats only address errors that we return from our application. Your API or intermediate HTTP proxy servers will always return HTTP 500 errors if a error occurs during transit.

See examples of the new error format




Data Examples


Appendix vs Inline Ancillary data

Note how in the inline format, there are full airport and carrier structures, but in the appendix format there are only FsCode references, but the appendix contains the fully structures.


Appendix format Inline format
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<response>
  <flightStatuses>
    <flightStatus>
      <flightId>268195273</flightId>
      <carrierFsCode>UA</carrierFsCode>
      <flightNumber>416</flightNumber>
      <departureAirportFsCode>LAX</departureAirportFsCode>
      <arrivalAirportFsCode>JFK</arrivalAirportFsCode>
      <departureDate>
        <dateLocal>2012-07-09T11:27:00.000</dateLocal>
        <dateUtc>2012-07-09T18:27:00.000Z</dateUtc>
      </departureDate>
      <arrivalDate>
        <dateLocal>2012-07-09T20:01:00.000</dateLocal>
        <dateUtc>2012-07-10T00:01:00.000Z</dateUtc>
      </arrivalDate>
      <status>L</status>

      ...........

   </flightStatus>
  </flightStatuses>

  <appendix>
    <airlines>
      <airline>
        <fs>UA</fs>
        <iata>UA</iata>
        <icao>UAL</icao>
        <name>United Airlines</name>
        <phoneNumber>1-800-864-8331</phoneNumber>
        <active>true</active>
      </airline>

      ...........
      
    </airlines>

    <airports>
      <airport>
        <fs>LAX</fs>
        <iata>LAX</iata>
        <icao>KLAX</icao>
        <faa>LAX</faa>
        <name>Los Angeles International Airport</name>
        <street1>One World Way</street1>
        <street2/>
        <city>Los Angeles</city>
        <cityCode>LAX</cityCode>
        <stateCode>CA</stateCode>
        <postalCode>90045-5803</postalCode>
        <countryCode>US</countryCode>
        <timeZoneRegionName>America/Los_Angeles</timeZoneRegionName>
        <localTime>2012-07-10T13:36:14.982</localTime>
        <utcOffsetHours>-7</utcOffsetHours>
        <latitude>33.943399</latitude>
        <longitude>-118.408279</longitude>
        <elevationFeet>126</elevationFeet>
        <classification>1</classification>
        <active>true</active>
      </airport>

      ...........

   </airports>
 </appendix>

    
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<response>
  <appendix/>
  <flightStatuses>
    <flightStatus>
      <flightId>268195273</flightId>
      <carrier>
        <fs>UA</fs>
        <iata>UA</iata>
        <icao>UAL</icao>
        <name>United Airlines</name>
        <phoneNumber>1-800-864-8331</phoneNumber>
        <active>true</active>
      </carrier>
      <flightNumber>416</flightNumber>
      <departureAirport>
        <fs>LAX</fs>
        <iata>LAX</iata>
        <icao>KLAX</icao>
        <faa>LAX</faa>
        <name>Los Angeles International Airport</name>
        <street1>One World Way</street1>
        <street2/>
        <city>Los Angeles</city>
        <cityCode>LAX</cityCode>
        <stateCode>CA</stateCode>
        <postalCode>90045-5803</postalCode>
        <countryCode>US</countryCode>
        <timeZoneRegionName>America/Los_Angeles</timeZoneRegionName>
        <localTime>2012-07-10T13:36:47.938</localTime>
        <utcOffsetHours>-7</utcOffsetHours>
        <latitude>33.943399</latitude>
        <longitude>-118.408279</longitude>
        <elevationFeet>126</elevationFeet>
        <classification>1</classification>
        <active>true</active>
      </departureAirport>

      ..........

    </flightStatus>
  </flightStatuses>
 </response>


    
back to examples





Response with contextual request information


See how the response contains a request section with explicit details on each parameter that influenced the response.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<response>
  <request>
    <url>https://api.flightstats.com/flex/flightstatus/rest/v2/xml/airport/status/JFK/arr/2012/7/9/20</url>
    <airport>
      <requestedCode>JFK</requestedCode>
      <fsCode>JFK</fsCode>
    </airport>
    <date>
      <year>2012</year>
      <month>7</month>
      <day>9</day>
      <interpreted>2012-07-09</interpreted>
    </date>
    <hourOfDay>
      <requested>20</requested>
      <interpreted>20</interpreted>
    </hourOfDay>
    <numHours>
      <requested>1</requested>
      <interpreted>1</interpreted>
    </numHours>
    <utc>
      <requested>false</requested>
      <interpreted>false</interpreted>
    </utc>
    <codeType/>
    <maxFlights>
      <requested>5</requested>
      <interpreted>5</interpreted>
    </maxFlights>
    <extendedOptions/>
  </request>
  <appendix>
    <airports>
      <airport>
        <fs>JFK</fs>
        <iata>JFK</iata>
        <icao>KJFK</icao>

        .............

      </airport>
    </airports>
 </appendix>
 <flightStatuses>
  <flightStatus>
      <flightId>268195273</flightId>
      <carrierFsCode>UA</carrierFsCode>
      <flightNumber>416</flightNumber>
      <departureAirportFsCode>LAX</departureAirportFsCode>

      .............

    </flightStatus>
  </flightStatuses>
</response>

back to examples

Track response with no flights and airport information


Note that there are no current flights in the air from Glasgow, but the request contains the validated airport code with the "fsCode" key. The appendix contains the fully-populated airport structure.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<response>

  <request>
    <url>https://api.flightstats.com/flex/flightstatus/rest/v2/xml/airport/tracks/GLA/dep</url>
    <airport>
      <requestedCode>GLA</requestedCode>
      <fsCode>GLA</fsCode>
    </airport>
    <includeFlightPlan>
      <requested>false</requested>
      <interpreted>false</interpreted>
    </includeFlightPlan>
    <maxPositions>
      <requested>2</requested>
      <interpreted>2</interpreted>
    </maxPositions>
    <maxPositionAgeMinutes/>
    <codeType/>
    <maxFlights>
      <requested>5</requested>
      <interpreted>5</interpreted>
    </maxFlights>
    <extendedOptions/>
  </request>

  <appendix>
    <airports>
      <airport>
        <fs>GLA</fs>
        <iata>GLA</iata>
        <icao>EGPF</icao>
        <name>Glasgow International Airport</name>
        <city>Glasgow</city>
        <cityCode>GLA</cityCode>
        <stateCode>SC</stateCode>
        <countryCode>GB</countryCode>
        <timeZoneRegionName>Europe/London</timeZoneRegionName>
        <localTime>2012-07-11T01:57:55.008</localTime>
        <utcOffsetHours>1</utcOffsetHours>
        <latitude>55.864213</latitude>
        <longitude>-4.431782</longitude>
        <elevationFeet>26</elevationFeet>
        <classification>2</classification>
        <active>true</active>
      </airport>
    </airports>
  </appendix>
  <flightTracks/>

</response>


back to examples

The new error reporting format


In this example, we've made a request with an airport code that doesn't exist. The HTTP response is 200, but the application response contains:
  • An error structure, including a numeric code, a message, and an error GUID
  • The request is detailed, include an error code attached to the parameter that failed validation
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<response>
  <error>
    <httpStatusCode>400</httpStatusCode>
    <errorCode>BAD_AIRPORT_CODE</errorCode>
    <errorId>a68cf1ba-1d64-4080-a3c4-848177fab36e</errorId>
    <errorMessage>The Airport code given did not match a known airport. Invalid airport code: 'XXX'</errorMessage>
  </error>
  <appendix/>
  <flightTracks/>
  <request>
    <url>https://api.flightstats.com/flex/flightstatus/rest/v2/xml/airport/tracks/XXX/dep</url>
    <airport>
      <requestedCode>XXX</requestedCode>
      <error>BAD_AIRPORT_CODE</error>
    </airport>
    <includeFlightPlan/>
    <maxPositions/>
    <maxPositionAgeMinutes/>
    <codeType/>
    <maxFlights/>
    <extendedOptions/>
  </request>
</response>



back to examples

Tuesday, July 3, 2012

What’s New with FlightStats Flex APIs

In response to your evolving needs, we’re introducing new Web services APIs. They
are code named Flex, reflecting our commitment to flexibility and innovation in the
flight status and tracking market.

For customers on our current platform, we’ll continue to support your integrations
into the future.


Some key differences:

First, positional data plays a prominent role in our new web services API offering.
We feature new ways to query, deeper tracking, and a richer data model. The
following table compares and contrasts positional improvements made over the
FlightHistory API.


Feature FlightHistory API Flex API
Lookup Vector Airline Code & Flight Number Airline Code & Flight Number
Airport Code
Airline Fleet
Lat / Long Bounding Box
Flight Plans No Yes
Number of Positions Last 200 Positions All Positions
International Flights Limited, US Airspace ADS-B

Second, flight status information is enhanced and extended. We’ve included many
of the commonly requested features in this redesign. The table below calls out a
number of the key improvements:

Feature FlightHistory API Flex API
Times Local Times
Offset to UTC
Local Times
UTC Times
Flight Type No Yes - Allows for identification of cargo flights
Linked Flights No Yes - Published Upline and Downline Flights
Delay Computations No Yes

Third, we’ve embraced a RESTful interface to our API while continuing to offer
XML and SOAP We now support JSON and JSONP responses making it easier to
develop AJAX style applications. In addition, we’ve simplified the requests, opting to
provide more endpoints while removing seldom used parameters.

Finally, we have implemented a new development center. After an analysis of the
work required to bring our API management consoles up to level we desire, we
decided to enlist the help of an SAAS API infrastructure vendor, 3Scale. Our focus is
on integrating and disseminating high quality flight information. 3Scale strengths
are the authentication, authorization, accounting and reporting infrastructure
required to run an API business. We’re proud of the partnership, new interface
and features for our development community. We’re anxious to bring our entire
offering to their platform.

We aren’t finished just yet. The Flex web services API is in beta and available to
you to explore and integrate. Our commitment to innovation will continue over
the coming months as we bring weather, on-time performance, flight alerts, data
feeds, and syndicated content on-line. We also have a rich roadmap of upcoming
enhancements to our core flight data product including irregular operation details,
predictive estimates, chance of delay and the ability to query flight status one year
in advance of the flight. To explore Flex … visit the new FlightStats developer center.