everythinglocation

[everythinglocation]

Outage Report (January 18, 2018) - https://saas.loqate.com

On January 18th, https://saas.loqate.com was unavailable via https://, http:// remained unaffected. This was due to a misconfiguration of auto-renewals for the *.loqate.com SSL certificates.

The service was affected from approximately 3:00 A.M. PST until 3:00 P.M. PST. https://api.everythinglocation.com remained unaffected.

[everythinglocation]

Service & Product Update (May 16, 2017)

Field status option - we’ve added a new server option that can be used with our /address/verify endpoint. When “FieldStatus“ is set, a second object will return within the output address record with component/compound fields and the respective field status code. This code is an extension of the AVC functionality that provides feedback on what process or change was performed on each address field. Documentation, along with a list/description of field statuses can be found on our website.

Q2 engine and data release - improvements with this release include new auto-complete functionality for Germany (supported countries now cover US, CA, AU, GB and DE), handling for sub-building data for United Kingdom addresses, and major country data reviews for Australia and Canada. There are no structural changes for either Local or Cloud APIs.

New status monitor - we’ll soon be launching some new visualization tools that provide an insight into the status, load, and traffic being served by our api.everythinglocation.com service. We’ll publish the URL to customers when ready.

[everythinglocation]

Outage Report (May 16, 2017)

Tuesday May 16, 2017

Impacted service(s): http(s)://saas.loqate.com Duration: unknown

Summary

Some requests using production API keys on our older saas.loqate.com platform were reported as returning a 500 Internal Server Error response code.

Immediately after the issue was detected, the affected server was rolled back to a previous release. On Wednesday May 17, a new, fixed instance was re-tested and moved to production.

Services provided through api.everythinglocation.com or api.loqate.com were not affected during this time. We recommend that any customers still using the older API migrate to the newer, more robust service as soon as possible.

Cause of the Outage

The error was caused by a small code change in the account management layer that wasn’t tracked in the respective sprint cycle. This had two consequences, a) the testing framework we use for updates didn’t include a test for this change, and b) the auto-test/pre-flight process we run on all instances prior to release didn’t include tests for this part of the code.

Remedial Action

Our response to this issue falls into two key areas:

1) Project and product teams have been reminded that any code change is to be conducted as part of the standard development/sprint process.

2) The auto-test system will be updated to include tests for any function that has the ability to restrict account access through the service APIs. This project is considered a mid-term improvement, and target delivery will be at the end of the current quarter.

[everythinglocation]

Service & Product Update (April 11, 2017)

Service/country restrictions for API keys - a significant new feature for partner and customers who access the cloud through GBG Loqate, our channel business. Any API key can now be restricted to a particular endpoint, add-on, and country (or countries/regions), making it easier to provide per-record pricing/access across specific API products and, where applicable, countries.

For example, if a key is locked to /address/verify for US and Canada, the response for any other combination (including add-on parameters) will return an error message.

At the moment, this feature is only being provided for high-volume customers and Loqate partners. Administering this feature will not be made available through the everythinglocation.com self-service interface.

New server options (advanced users only) - new options include DuplicateHandlingMask, EnhancedCountryTool, NativeAddressLineReverse, ReturnVerifiedFieldsOnly. These options have been exposed primarily for investigative and debugging purposes only. We do not recommend customers use server option within a production environment, unless the option provides specific and necessary functionality (such as transliteration or output casing). Documentation will be added to the main website based on the demand for the manipulations that these options provide.

Contact form fix - the order that scripts were loading was preventing our ReCAPTCHA module from loading on some browsers, therefore disabling our contact form. This has now been resolved in this update.

[everythinglocation]

Outage Report (April 7, 2017)

Date: Friday April 7, 2017

Impacted service(s): http(s)://saas.loqate.com Duration: 9:33AM - 10:40AM PT (± depending on DNS cache & propagation)

Summary

The domain name http(s)://saas.loqate.com was not resolving to any known host for some users. 

Cause of the Outage

The NS records for *.loqate.com were changed during an update to https://www.loqate.com via our domain registrar for *.loqate.com. This meant that our newer management system through the AWS Route53 service was completely circumvented and that DNS servers were pointing to our domain registrar instead. Unfortunately, it was out of our control as our older saas.loqate.com service still shares domain management with other teams within the business.

Our main API, api.everythinglocation.com, which is managed in a separate hosted zone, was unaffected during this time. 

Remedial Action

The following actions have been conducted:

Our monitoring service first saw an issue around 9:33AM, the DNS records were reverted back to their original state at 10:00AM. We’ve since learned that the original change causing the problem was performed between 8:30AM and 9:30AM.

We have consulted the team handling www.loqate.com and requested that any future changes on the loqate.com domain are first checked by the cloud infrastructure team.

If there are any questions regarding this service outage, please email hello@everythinglocation.com

[everythinglocation]

Important API Change for Auto-Complete Users

This message is only applicable to users of our AutoComplete API on api.loqate.com or api.everythinglocation.com.

In an upcoming update we’ll be making a small change to the metadata object that is returned with all queries sent to /address/complete. Currently, when the TotalRecordCount within a corresponding metadata element is greater than 1, the result will return all matching components and their values.

This is returned so that an application can send the fields and values back into the /complete endpoint as filters for the next query.

One of the fields returned today is Address, which is not a supported field in the input filters object, and using it may cause the /complete endpoint to behave incorrectly. Including this field in the product was not intended, and we’ll be removing it with our next release.

Current output, highlighting the field to be removed

"metadata": [

   {

     "TotalRecordCount": "2",

     "AdministrativeArea": "West Midlands",

     "Locality": "Birmingham",

     "Thoroughfare": "Arden Road",

     "Address": "Arden Road, Acocks Green, Birmingham"

   },…

If you are currently using this field within your application, please instead use the corresponding value in the output object, which should be the same.

Revised output after release

"metadata": [

   {

     "TotalRecordCount": "2",

     "AdministrativeArea": "West Midlands",

     "Locality": "Birmingham",

     "Thoroughfare": "Arden Road"

   },…

If this change requires additional development time to resolve, please let us know so we can help accommodate your time frames.

[everythinglocation]

Service & Product Update (February 28, 2017)

New Auto-completeJS website integration - you may have read a recent post about some major improvements that are now underway within our auto-complete API. To coincide with these exciting changes, we’ve completed the first version of our single-line JavaScript integration. Adding type-ahead functionality to an existing address line or new address search input field is now wrapped into an easy to implement, copy-paste-go script. More information, including a step-by-step integration guide, can be found on our site: https://www.everythinglocation.com/resources/api/autocompletejs/

Address engine quarterly release - In mid-January we updated our address processing engine (on saas.loqate.com and api.everythinglocation.com) to be in line with the GBG Loqate Q1.0 release. New features include parsing improvements for India, minor updates to the auto-complete search process (improved handling for sub-buildings, alphanumerical characters, and slashes and hyphens), and improved data alignment/verification uplift for Finland and Switzerland.

Subscription management - users can now change and cancel monthly credit subscriptions from their everythinglocation.com account settings.

Ending support for saas.loqate.com - last month we posted about migrating all users of our legacy saas.loqate.com system onto our newer, improved REST API hosted at api.everythinglocation.com. Although we will keep the older API active, we will be ending support and updates to the service from as early as July 1. We will be working directly with users who are yet to migrate over.

Updated documentation for reverse geocoding - you can find it here: https://www.everythinglocation.com/resources/api/get-address-reverse/. We’re grateful for the feedback from customers since releasing the new endpoint.

[everythinglocation]

Outage Report (February 14, 2017)

Date: Tuesday February 14, 2017 Impacted service(s): http(s)://api.everythinglocation.com Duration: 10:02 to 13:22 PT (3hrs 20mins)

Summary

We were alerted to disruptions on our new API service on Tuesday morning, and later experienced a full outage on services running on api.everythinglocation.com. By 13.22 PT, all services were fully restored.

Services provided through our legacy API hosted at saas.loqate.com were unaffected during this time.

Cause of the Outage

The deployment mechanism that initializes new instances within our auto-scaling groups experienced a resource include failure during a period of increased load. The failure prevented any new instances being created, and as a result all core servers began to slow and throttle load, resulting in request timeouts.

An instance reboot was attempted prior to discovering the primary cause, which then caused a full outage as the boot sequence was dependent on the deployment resource.

The origin of the issue was traced back to a dynamic variable in our deployment configuration that was no longer being supported because of a version update we hadn’t been made aware of.

Remedial Action

The following actions have been conducted:

When the root cause was discovered, our deployment configurations were updated to be in line with the latest resource version. New instances were then checked, tested, launched and then deployed to api.everythinglocation.com.

Following this initial fix on Tuesday, all cloud API instances were re-built on Wednesday using a static version of the deployment resource, therefore completely removing the risk of re-occurrence.

For all future releases, updates and tests will be performed on all dependencies within our deployment resource architecture. This process removes any dynamic references to external libraries.

If there are any questions regarding this service outage, please email hello@everythinglocation.com

[everythinglocation]

It's Time to Migrate to Our New API

As our new cloud platform scales and matures, we want to ensure that both new and existing customers receive the best in terms of product and performance. If your business is currently consuming our older web service hosted at saas.loqate.com, then please read on.

This post compares our newer API (hosted at api.everythinglocation.com) to our legacy service. We’d like to encourage all customers that are integrated with our saas.loqate.com API to migrate over to api.everythinglocation.com at the earliest convenience. Please note that we are planning to retire saas.loqate.com by mid-2017.

New API Features (Summary)

api.everythinglocation.com is deployed in a reliable, auto-scaling environment that provides load balancing and redundancy across multiple server locations.

API services are managed through a homegrown continuous deployment framework, which includes auto-testing, load testing and a regression testing platform.

We’ll soon launch our account management API, which allows programmatic access for advanced or custom interface development.

Our new API provides automated email notifications for credit-based accounts when credit is running low or expiring.

We’ve built our new API to comply with the latest RESTful standards, and have improved our error messaging framework.

We’re only launching new products (such as email validation and reverse geocoding) and features through our new API.

Products Descriptions

Below, we present our new API products. For address validation and geocoding, the functions performed by the address quality engine are the same, meaning a record passing through both old and new API will yield the same result.

All products served through the new API will perform better in terms of load capacity and improved response times.

Address Verification (/address/verify plus add-on services)

The endpoint accepts an input object in JSON or XML format that can accept up to 200 address records for single record and batch processing.

Input fields now use full field names, and are no longer restricted to addr, add1…add6, lcty, admn, ctry, and pcde. All available address fields can be used.

Customizable processing options are sent within an options object. Defaults for some server options are set on the API, which can be overridden for custom workflows.

Enrichment services are requested by parameter, and include geocode, suggest, certify and enhance.

The new API service no longer supports GET requests.

Address Auto-complete (/address/complete and /address/capture)

This product differs from the older search functionality (formerly called using the parameter p=s). We’ve taken the Loqate Auto-complete and Powersearch products to create a session-less REST interface that provides suggestions as the user types, returning a full address profile when an address is selected.

The product is best suited for interactive, single-field address capture interfaces. Optimal results are generated with partial input.

Enrichment services are requested by parameter, and include geocode, certify and enhance.

Search quality and performance has recently been improved for US, UK, CA and AU. In these countries the auto-complete service matches on multiple fields and provides a filter (or drill-down).

In addition, we have recently launched a JS application that can be imported into web forms with minimal effort.

Reverse Geocode (/address/reverse, documentation coming soon)

The product accepts a digital longitude and latitude and returns a candidate list of nearby address records.

Email Validation (/email/verify and /email/batch)

Verify email addresses in real-time (per record) or through our file upload API.

Product Comparison & Mapping

The products served through the older API map across to the new API as follows.

Products Not Migrating Over

Our REST Batch (file upload) API.

Due to a shift in demand to REST, our SOAP API will not be migrated.

REST service endpoints, such as our reporting and authentication APIs.

Verification processes, including Query (p=q), Parse (p=p), Match (p=m) and Format (p=f). These are not planned for migration.

Pricing Benefit for Credit-Based Charging

Most self-service cloud accounts now operate using our new credit charging model. Credits are our universal currency that provide a convenient and flexible way to access all of our API products and tiers.

With the new API, we’ve made the charging model more attractive to the most widely used countries – the chart below compares the product and credit cost between new and old APIs.

For account managed and high-volume accounts (served through GBG Loqate), cloud credits and credit charges will not apply.

What About My API Keys and Existing Cloud Credit?

Our product API layer sits separately to our account management layer and account UI. This basically means that your credit and API keys work seamlessly on both old and new APIs. Please note though that each API charges differently, as detailed above.

Ready to Switch Over?

Okay, maybe not quite yet. We appreciate there are a few technical changes that need to be both evaluated and scheduled into a release or development cycle.

Full documentation and support are offered on our website, and we encourage you to start implementing the new endpoints as soon as you can. If you have questions or need help, we recommend that you contact our cloud team or your account manager to start this process right away.

We plan to end support for our older saas.loqate.com API by mid-2017.

[everythinglocation]

Recent API Updates to the Everything Location Auto-Complete Service

Summary

In this update we’ll run through some changes in the Everything Location cloud API that reflect the start of a longer-term search process improvement project. Since launching the changes, documentation to access the new features has remained in draft while some additional changes are performed on the API. This post will step through the changes and provide some guidance on how to create an auto-complete workflow using the /complete and /capture endpoints.

Background

In October 2016, GBG Loqate, who both operate and supply Everything Location services, delivered a significant update to the search functionality that powers the auto-complete service. In the initial release, the new functionality could only be used for address searches conducted on United States, United Kingdom, Canada and Australia data. To support the improvements, some additions were made to the /complete endpoint.

Search on Any Field

The first big improvement allows the search process to match the input query to any component field rather than just the start of a compound address field. This means a user could receive a meaningful set of suggestions by entering different parts of the address. For example, the input query ‘Burlingame’ would present both complete and component-matched records where the string was present in different parts of the address.

Filter-Based Searching

For results that may represent only a partial match to multiple addresses, the matched field(s) can be sent back into the /complete endpoint to narrow or filter any subsequent queries. For example, ‘Palo Alto‘ will return ‘Palo Alto, CA’ as a suggestion. A new ‘metadata’ object for this value will be returned containing the match fields (in this case Locality=Palo Alto and AdministrativeArea=CA) and the number of full address records that would match these values together (’TotalRecordCount’). If a user selects ‘Palo Alto, CA’ the next call to the /complete endpoint would pass back ‘Locality‘ and ‘AdministrativeArea‘ along with their values within the filter object. Any new queries will then be filtered by these values.

Workflow

If using the auto-complete endpoints to create a custom end user experience, the new functionality can be engineered in the following way.

The metadata (output) and filter (input) objects have no defined structure. The metadata result will only contain the fields that are matched plus the TotalRecordCount field. If the selected record cannot filter any further (i.e. it is a completed address), then only the TotalRecordCount field will return with the value of 1.

Rapid Integration for Web Forms

A JS application will be published shortly that will allow the above functionality to be quickly integrated into web forms.

Expanding the New Functionality

The two new features above (Including the metadata and filter objects) are currently only functional on United Kingdom, United States, Canada and Australia data. That doesn’t mean customers need to create a separate integration. For other countries, the functionality only returns completed address records, so the metadata will always and only have TotalRecordCount=1.

During the year, further countries will be added, starting with Western Europe in H1. As soon as new countries are added, we’ll update our service blog.

[everythinglocation]

Outage Report (January 20, 2017)

Date: Friday January 20, 2017 Impacted service(s): https://saas.loqate.com (only HTTPS) Duration: 03:30 to 07:25 PT (3hrs 55mins)

Summary

The outage only affected HTTPS requests made to our old API (saas.loqate.com). All other services were unaffected. The cause of the downtime was a SSL certificate mismatch following the release of the Loqate Q1 update to our cloud API. This has been tracked to a routine check on our deployment service.

Resolution

The development steps required to remove the point of failure in the deployment workflow have been identified and scheduled to be resolved prior to the next scheduled Loqate minor release in mid-February. Additional connectivity testing will also be performed on all future engine updates.For any questions relating to this outage report, please use the contact form on everythinglocation.com

[everythinglocation]

Service Update (November 22, 2016)

Reverse geocoding API - a quick update to our reset password and confirm email address emails to bring added consistency to all system-generated messages.

API whitelisting (domain name) - a quick update to our reset password and confirm email address emails to bring added consistency to all system-generated messages.

[everythinglocation]

Service Update (September 12, 2016)

In this post we’ll provide a quick summary of any changes or new features hosted through api.everythinglocation.com, saas.loqate.com, and the Everything Location website.

New email templates - a quick update to our reset password and confirm email address emails to bring added consistency to all system-generated messages.

Credit expiration gap - additional overlap is now offered to monthly subscription plans so that expiration will not occur after the time that the subscription triggers.

Transliteration - new OutputScript parameter is available for the /address/verify/ endpoint. If the country supports one of the native scripts (listed below), then setting the value to the native ISO/IEC 8859-7 code will transliterate the output record. If the parameter is omitted, the value defaults to match the character set used with the input record.

Available character scripts and countries include: LATN (Latin), CYRL (Cyrillic, Russia), GREK (Hellenic, Greece), HEBR (Hebrew, Israel), HANI (Kanji, Japan), HANS (Simplied Chinese, China), ARAB (Arabic, UAE), THAI (Thai, Thailand), HANG (Hangul, South Korea).

[everythinglocation]

Service Update (August 31, 2016)

This post covers a quick summary of any changes on our older API hosted at saas.loqate.com.

Remote file server migration - impacting only the Batch UI and REST API, we’ve moved our file server from our local data center into AWS. The move provides added reliability and security to both inbound and outbound data being processed through the batch system.

As a part of this process, we have begun to archive output data files. Moving forward, any output files older than six months will not be available to download via the API or website.

However, any customers requiring to download and access output files that were generated more than six months ago should contact our development team.

Report API fix - following a database configuration change in early August, usage data for some accounts was not being loaded through the Report endpoint. This issue is now fixed and all usage data will be returned/displayed.

Loqate version update - both data and engine are in line with the Q3 Loqate release (2.20.0.8559). This update was released a little later than the scheduled service release due to other project dependencies. Both saas.loqate.com and api.everythinglocation.com are now current and on the same version.

[everythinglocation]

Service Update (August 30, 2016)

In this post we’ll provide a quick summary of any changes or new features hosted through api.everythinglocation.com, saas.loqate.com, and the Everything Location website.

Credit balance breakdown - customers using the self-service interface can now view a breakdown of how their credit balance is made up. This is primarily a usability feature that is helpful if an account is using multiple credit packs or plans with different expiration dates.

IP white-listing for API keys - the new API can now provide security for customers that use API keys within a client side environment.

One or more IP addresses can be saved to any API key which will be checked against the source of an API request. If the values don’t match, the request will be rejected. One of our next projects will be to provide a self-service interface via the account settings so users can control the security settings on individuals keys. Until then, customers can contact us to set up IP address(es) restrictions on their key(s).

Messaging on payment response cases - this a small fix to provide improved messaging back to the user interface in the case where payments through our Stripe integration have failed or cannot be processed.

[everythinglocation]

Timeout Issues (w/c August 29, 2016)

Following a few reports that a handful of transactional requests into saas.loqate.com endpoints have been timing out, we have expanded our server pool to provide a greater capacity for inbound connections.

Since the outage experienced on saas.loqate.com in July, the Everything Location team have been busy migrating all remaining processes and assets from our local data center on to AWS. Part of this project involves moving a proxy/relay that currently handles all saas.loqate.com traffic. Once done, the server capacity will greatly improve. Conclusion of this work will be reported within a future service update (expected early next quarter).