espinr - Tumblr blog

espinr · 6 years ago

Text

Nuestro Capitán

[Esta columna de opinión fue publicada, truncada, por La Nueva España el 30 de junio 2019, acompañando a la noticia sobre el homenaje a José Luis Capitán en la cima de El Angliru. Ésta es la versión completa:]

‘Capitán’.- persona que encabeza un grupo o movimiento humano. Esa es la definición oficial de la Real Academia Española que me sirve para comenzar a definir a José Luis, cuyo nombre está marcado como homenaje eterno en una de las cumbres más emblemáticas y famosas de nuestra región, El Angliru. Capitán es su apellido, célebre en el mundo del atletismo de élite y las carreras populares. Es la voz y la cara más conocida del ‘running’ asturiano y ha influido de una u otra forma en la vida de muchos asturianos, sean deportistas habituales o no.

José Luis, madrileño de nacimiento, comenzó sus andaduras atléticas en la adolescencia siguiendo los pasos de su hermano Vicente. Su primer día de entrenamiento ya lo podemos considerar como muestra de su tenacidad y entusiasmo: como hacía frío y llovía estuvo una hora corriendo alrededor de la mesita de la sala de estar de su casa. Pasadas las agujetas, y mejorado el tiempo, comenzó a entrenarse con Isidro Rodríguez, en un grupo que juntaba a muchos de los grandes del atletismo madrileño, generando centenares de anécdotas y entrenamientos de calidad, que aún perduran en la historia del Parque de Arcentales.

El impacto de José Luis en el deporte asturiano ya se comenzó a forjar en las visitas esporádicas cuando aún vivía en Madrid. durante los fines de semana en Asturias, sacaba sus dotes de líder y espontáneamente movilizaba a docenas de atletas asturianos para disfrutar de entrenamientos extenuantes. Siempre con ideas disrruptivas para mejorar el futuro del atletismo, pensando en involucrar a más jóvenes y adultos en la práctica del atletismo preservando los valores deportivos y sociales. Ha inhalado deporte y exhalado educación, organizado decenas de eventos por el simple hecho de llenar esos lugares vacíos de deporte, atrayendo a las nuevas generaciones a este mundo de valores encomiables. Visionario, en 2012 organizó un Campeonato de España de carreras por montaña en Tineo, para promocionar la disciplina y presumir de la tierra de su mujer, Teresina, como la llama cariñosamente.

De Capitán, como atleta, siempre he admirado su versatilidad. Rápido en la pista, buen corredor de asfalto y amante del campo a través y la montaña, consiguiendo medallas nacionales y algún Campeonato de Asturias. Pero más allá de lo deportivo, admiro su pasión, compañerismo y entrega con total humildad. “Aquel élite” madrileño que nos visitaba para arrasar en las carreras populares asturianas nos cautivó a todos. Competitivo, pero siempre ponderando más el trabajo del resto que el suyo propio. Su ligereza y su fuerza le hacían imbatible en pruebas de gran dureza como era la Subida al Naranco o en cualquier media maratón. En 2003, conoció El Angliru y ahí empezó a forjarse la leyenda.

El Angliru, el olimpo del ciclismo, marcó un hito también en las carreras pedestres a nivel nacional. Visto como un reto con la naturaleza más que una competición entre humanos, ha reunido a miles de atletas de toda la geografía nacional y enfrentado a los mejores escaladores a pie con sus brutales desniveles. José Luis fue uno de los que se enamoró a primera vista del gran coloso. Dos años después del primer intento, volvió a subir y consiguió un récord que aún se mantiene.

Una década después de la primera edición, con conflictos organizativos, y siendo una prueba abocada a su desaparición, Capitán tomó las riendas de la misma con esfuerzo personal (y familiar) y reinventó el reto, atrayendo tanto a ciclistas como atletas, haciendo de la subida al Angliru una fiesta de confraternización entre corredores, familias y la montaña riosana. Desde entonces, ha continuado su lucha por preservar estos valores ligados al deporte y la educación, dando el máximo de sí mismo, bien sea corriendo, gestionando o animando.

Este homenaje es algo anecdótico pero necesario. Somos muchos los atletas y curiosos del atletismo que siempre hemos rendido veneración en silencio cuando oímos hablar de El Angliru o de Capitán, que ya son sinónimos. Espero que esta referencia física en la cima permita a que muchos profanos que la vean se interesen por estas historias, hablen con él, y se contagien de su fuerza y entrega. Batir su marca está al alcance de muy pocos, pero será más difícil superar lo que está haciendo por todos nosotros. Necesitamos más Capitanes en este mundo.

#capitán #running #atletas #atletismo #Asturias #angliru

0 notes

espinr · 6 years ago

Photo

Alter ego (y mis Brooks Match 17) 💙. @larus_lara, es brutal... Gracias!! 🙇🏻‍♂️ #runner #actionfigures https://www.instagram.com/p/Bwe5SQiF8Zy/?utm_source=ig_tumblr_share&igshid=15olplgbpygox

#runner #actionfigures

0 notes

espinr · 7 years ago

Text

The Streets of Gijón: from the Newspaper Archives to the Knowledge Graph

Read this article in Spanish

Mis Calles Gijón/Xixón is a Web application started as a personal project, motivated to support my curiosity when walking in the streets of my hometown. I always wonder who was the person, historical fact or place that shaped cities and understand the motivations to name streets after them, as part of a kind of tribute to preserve the history of places.

Technologically, Mis Calles is a Progressive Web Application, suitable for any mobile device, enabling the use of their location capabilities to enrich the user experience. The application was developed through a series of complex Python scripts, which allowed to automate the ingestion and processing of the information, coming from open sources. This automation is interesting since we could deploy a Mis Calles in any city in the world with just minimal adjustments.

The development process had the following steps:

Step 1: The streets and their location

The City of Gijón, through its Transparency and Open Data Portal, publishes two datasets that served to: (1) identify the streets of Gijón —list of streets, type (e.g., avenue, road, park, street, etc.), unique names (no acute accents) and numerical identifiers—, and (2) locate them on a map. The location of the streets is based on the dataset of streets and portal numbers, a table with thousands of coordinates corresponding to each street number (building) and street in the city (not the line or polygon).

Step 2: Generate the polygons on the map

After nesting the tables of streets and numbers, another script built the lines and polygons (in KML and geo-JSON format) shaping the streets, enabling the visualization on a map.

For this, each street is divided into odd and even street numbers. The odd ones are ordered in ascending order and a polygon is generated by joining all the points that form the coordinates. Once the script reached the biggest, it joins all the even points in a descending way until reaching the lowest. Finally, the polygon is closed linking the first point of the list.

This technique is precise in certain ways, but if the street has few registered portals it can give rise to polygons with curious shapes, as can be seen in Calle Mariano Pola, below.

Step 3: Description of streets I: Piñera method

Once the streets were identified univocally, including their approximate location, the streets were described based on the El Comercio's publication: The streets of Gijón, History of their names, a great work by Luis Miguel Piñera.

The portal offers open access to this publication but it does not specify reuse conditions. After several formal use requests to the City Council, and receiving no response (I am still waiting), I asked directly to El Comercio. They quickly agreed and wanted to take part in this project.

In early 2000, you would not expect more than reading that publication as PDF, so no efforts were made beyond the PDF file. With a scraping script, I automated the extraction of the main data of the work: a name of the person or event that named after the street, description, date of registration and previous names. The extraction was chaotic (e.g., changes in the expected structure, page breaks, figures, cites, etc.), and the result had to be double checked almost manually.

Another Python script nested the resulting tables: list of streets, geo-localized and now including a description. In the case of not finding the exact match between the denomination (e.g., comparison of 'LOS CALEROS' and 'CALEROS, LOS'), the script prompted asking for the right choice.

Step 4: Description of streets II: Knowledge Graph

Although the real value of the application comes from Piñera's historical investigation, several streets were not included. In this case, these entities are usually concepts or contemporary people that can be easily recognizable and whose descriptive information can be found on the Web. So, another method of finding information associated with those entities was searching the Knowledge Graph, which Google offers through an API.

The Knowledge Graph is a structure of information linked by semantic concepts —is what Google uses to show proper results. This graph contains information from open sources, such as Wikipedia. It includes metadata associated with the concepts that characterize them in depth.

This data source is really interesting to get information about places, events or people. The script queries the API recognizing the types of results obtained. The results also show the degree of success confidence. So in the case of finding a match of the expected type, the script stores the data in a table with the needed information: description, alternative title, URL of the entity (Wikipedia page), image URL and associated license, standardised entity type using schema.org (eg, CivicStructure or LandmarksOrHistoricalBuildings). For example, the result obtained for the entity'EUROPA' [Square], is:

Name: Europa

Score (confidence): 244.048874

URL: https://es.wikipedia.org/wiki/Europa

Type: Continent, Thing, Place, AdministrativeArea

Description: Europe is one of the continents that make up the Eurasian supercontinent, located between the parallels 36º and 70º north latitude.

Imagen: http://t0.gstatic.com/images?q=tbn:ANd9GcS0uZNMs4cfxsd-XAvy8iNVY-6CjX9gMCUV2BSloYIjgEQewznu

Image license: https://en.wikipedia.org/wiki/Wikipedia:Text_of_Creative_Commons_Attribution-ShareAlike_3.0_Unported_License

Knowledge Graph URI: kg:/m/02j9z

Unfortunately, Google has removed the dereference and visualization of the Knowledge Graph entities —a few weeks ago, we could visualize an infobox with the information related to the entity. Anyway, this does not affect the application.

Step 5: Types of Entities

The Knowledge Graph, and its queries were the basis of the classification of the entities in an automated way. In the previous example, Europe is of type: Thing (thing, of course); Continent (continent); Place (place); and AdministrativeArea (administrative area).

Approximately, the 40% of the entities represented were classified automatically, always under human supervision, since there are confusing cases in which artificial intelligence can not do anything.

For example, Calle 'Cabrales' refers to the concept of 'Cabranes'. Within the Knowledge Graph, both concepts are identified and described as municipalities in Asturias, but actually, the street is named after a person.

Once all the tables were joined and nested (listing, geo-positioning, descriptions with the Piñera method and the Knowledge Graph, and types) I completed the classification using a subsequent semi-automatic homogenization of the types. For this, I used OpenRefine.

Finally, all the themes were defined in an intuitive taxonomy through a card sorting exercise. As a result, 1035 entities were grouped as:

(58) Artists (musicians, painters, actors, athletes);

(53) Concepts (feelings, jobs, physical elements)

(11) Explorers (historic conquerors)

(220) Geography (municipalities of Asturias, geographical features, populated places, etc.)

(113) History (historical events, people, landmarks of historical relevance)

(69) Business (people related to industrial and/or urban development)

(112) Letters (writers, thinkers, philosophers, etc.)

(105) Nature (fauna, flora, and other natural concepts)

(21) Nobility (monarchs and nobility in general)

(27) Organisations (companies and organizational entities)

__(94) Politicians (governors, military and councilors)

(79) Religion (religious people, places and concepts)

(73) Scientists (mathematicians, physicists and science or technology persons)

Also, taking advantage of other concrete types gathered by the scripts, I produced a second level of topic themes. For example, the subject Geography includes the following sub-themes: Country (27 entities), Geography of Asturias (66), Geography of Spain (44), Geography of Gijón (21), International Geography (13), Historical Places (1), Orography (48).

Have a look at the result of Mis Calles Gijón/Xixón, play with it and send me your comments.

#opendata #knowledge graph #tourism #history #local #people #streets #kg #gijon #xixon #webapp #semantic web

0 notes

espinr · 7 years ago

Text

Conoce tu ciudad a través de “Mis Calles Gijón/Xixón”

Mis Calles Gijón/Xixón es una aplicación Web que pretende mantener viva la historia de Gijón a través de los personajes, industrias o acontecimientos que han dado nombre a sus calles. Desde Gigia hasta Quini, pasando por Jovellanos y la Revolución Industrial y su carbón.

Este proyecto surge tras una colaboración entre el Diario El Comercio, cuya hemeroteca tras 140 años es la mejor base histórica para el contenido que muestra la aplicación, y CTIC Centro Tecnológico. Hemos implementado la plataforma y hemos sido capaz de realizar una extracción y publicación de datos de forma automatizada, usando tecnologías de inteligencia artificial y basándonos en datos de repositorios públicos.

La aplicación recopila más de un millar de calles, organizándolas por el tipo de entidad que le da nombre: Artistas (pintores, músicos, deportistas, etc.); De Letras (literatos y pensadores); Industriales (personas relacionadas con el desarrollo industrial o urbanístico); Nobles; Polític@s (gobernantes, militares y administradores públicos); Científic@s; Organizaciones; Religión (personas o lugares relacionadas con la religión); Exploradores; Historia (eventos y lugares históricos); Naturaleza (fauna, flora y otros conceptos naturales); Geografía; y otros Conceptos (p.e., sentimientos y oficios).

Ejemplo de concentración de dos tipos de entidades sobre el mapa

La base de datos de vías urbanas procede del Portal de Transparencia y Datos Abiertos del Ayuntamiento de Gijón. La clasificación y descripción de cada uno de los conceptos que dan nombre a las vías se ha realizado tomando dos fuentes de datos principales: (1) la publicación de El Comercio, Las calles de Gijón, Historia de sus nombres, del historiador Luis Miguel Piñera; y (2) del Knowledge Graph (Grafo de Conocimiento), una base de datos de fuentes abiertas que enlaza los conceptos semánticos y permite el procesamiento inteligente de la información. Gracias al Knowledge Graph también se han podido encontrar y extraer imágenes representativas para la mayoría de las entidades.

Mis Calles Gijón/Xixón permite la detección automática de la localización de los usuarios para sugerir las calles cercanas y permitir realizar búsquedas basadas en la situación en cada momento. La navegación por la aplicación es simple, permitiendo a los usuarios ver y filtrar los resultados en función de sus preferencias a través de tres vistas diferentes: sobre un mapa, con agrupaciones de los tipos de entidades por colores; en una lista simple; y en fichas descriptivas individuales.

Vista en forma de fichas

Un buscador, permite visualizar calles específicas en función de su nombre y una pestaña en la parte inferior permite a los usuarios ver qué calles están a su alrededor.

Calles cercanas a la posición actual

Información detallada

Las fichas correspondientes a cada calle contienen una imagen ilustrativa, el título oficial y un nombre alternativo (p.e., [Calle] Arrieta, Emilio Arrieta), la descripción correspondiente a la publicación de El Comercio y un resumen de la entrada de la Wikipedia (si fuese posible). Un desplegable en la ficha permite visualizar más información sobre la entidad que se muestra (nombres anteriores o comentarios de los usuarios), ofreciendo enlace directo a la hemeroteca o a la Wikipedia, así como verla situada sobre un mapa de la ciudad.

Un icono con el logo de Gijón, permite identificar a aquellas entidades relacionadas directamente con la ciudad (personas que han vivido en Gijón o han influido directamente sobre la ciudad).

La aplicación sugiere relaciones entre las personas o cosas que se muestran. Por ejemplo, para el Diario El Comercio, se muestran periodistas o personas que dan nombres a otras calles y están relacionadas con el periódico.

Colaboración ciudadana (Crowdsourcing)

Debido a que la mayoría de los datos han sido captados de forma automática, es posible que las descripciones contengan errores o, incluso, las imágenes no se correspondan con lo que pretenden representar. Por esto, la aplicación habilita un mecanismo de reporte de comentarios, errores o sugerencias con el objetivo de mejorar la aplicación con el conocimiento experto de aquellas personas que conocen detalles que merecen ser contados y recordados, o poseen fotografías que ilustran mejor la historia que se quiere contar.

Prueba ya la aplicación y coméntanos qué te parece. Si deseas conocer más detalles técnicos sobre la implementación, echa un vistazo a este blog post sobre cómo se ha construido Mis Calles Gijón/Xixón.

#miscalles #elcomercio #gijón #xixón #webapp #development #historia

0 notes

espinr · 7 years ago

Text

Mis Calles Gijón/Xixón: de Piñera al Knowledge Graph

Read this article in English.

La aplicación Mis Calles Gijón/Xixón surge de un proyecto personal para dar soporte a la curiosidad, que a muchos de nosotros nos despierta, el callejero y los personajes y hechos históricos que dan nombre a las ciudades y preservan su historia a través de estos homenajes ofrecidos por su ciudadanía y gobiernos.

Tecnológicamente, Mis Calles es una Aplicación Web Progresiva, adecuada a cualquier dispositivo móvil y que utiliza sus capacidades de localización para enriquecer la experiencia de los usuarios. La aplicación ha sido desarrollada mediante una serie de complejos scripts escritos en Python, que han permitido automatizar la ingesta y procesamiento de los datos, procedentes de fuentes de datos abiertos. Esto es interesante, ya que podríamosdesplegar un Mis Calles en cualquier ciudad del mundo con unos ajustes mínimos.

El proceso de construcción ha sido el siguiente:

Paso 1: Las calles y su localización

El Ayuntamiento de Gijón, a través de su Portal de Transparencia y Datos Abiertos, publica dos conjuntos de datos que me han servido para: (1) identificar las Calles de Gijón —listado delas calles, tipo de vía (avenida, camino, calle, etc.), nombre únicos (sin tildes) y con identificadores numéricos—, y (2) localizarlas sobre un mapa. La localización de las calles se hace basándose en el conjunto de datos de Calles y números de portales, una tabla con miles de coordenadas correspondientes a cada portal de cada calle de la ciudad.

Paso 2: Generar los polígonos sobre el mapa

Tras anidar las tablas de calles con la de los números, se generó otro script encargado de construir las líneas y polígonos (en formato KML y geo-JSON) que forman las calles y posteriormente permitirán determinar si nos encontramos en una calle u otra, así como visualizarlo en un mapa.

Para ello, cada calle se divide en número de portal pares e impares. Los impares se ordenan de forma ascendente y se genera un polígono uniendo todos los puntos que forman las coordenadas; al llegar al mayor se une con las coordenadas del mayor de los pares y luego uniendo todos los pares de forma descendente hasta llegar al menor. Finalmente, se cierra el polígono con el primero de los puntos.

Esta técnica es precisa en ciertas vías, pero si la calle tiene pocos portales registrados puede dar lugar a polígonos con formas curiosas, como se puede ver en la Calle Mariano Pola, a continuación.

Paso 3: Descripción sobre las calles I: método Piñera

Una vez identificadas las calles unívocamente, incluso su localización aproximada, se dota de descripción a las calles hemos utilizado una publicación editada por El Comercio y distribuida en abierto por el propio Ayuntamiento: Las calles de Gijón, Historia de sus nombres, una gran obra de recopilación histórica de Luis Miguel Piñera.

Desde el portal se ofrece derecho de acceso pero no se especifican condiciones de reutilización, así que tras solicitar repetidamente su uso formalmente al Ayuntamiento y no recibir respuesta (sigo esperando), se hace la solicitud a El Comercio, quienes acceden encantados.

Ya que a principios del 2000 no se esperaría otra cosa que leer el PDF, así que tampoco se hicieron esfuerzos más allá que conservar el archivo PDF. Con un script se automatizó la extracción de los datos principales de la obra: título de la entidad que da nombre a la calle, descripción, fecha de registro y nombres anteriores. La extracción dio lugar a errores, debido a cambios en la estructura esperada, como los saltos de página por ejemplo. Hubo que hacer un repaso manual para evitar textos fuera de lugar, como pies de foto u otros elementos como referencias o números de página intercalados en las descripciones.

Otro script juntó las distintas tablas: listado de calles, geo-localizadas y ahora con una descripción. En el caso de no encontrar la coincidencia exacta entre la denominación de las entidades (p.e., comparación de ‘LOS CALEROS’ y ‘CALEROS, LOS’), se le ofrecía al humano potenciales coincidencias.

Paso 4: Descripción de las calles II: Knowledge Graph

Aunque el valor real de la aplicación viene de la investigación histórica de la publicación de Piñera hay ciertas calles, cuyas entidades no aparecen en la publicación. En este caso suelen ser conceptos o personas contemporáneas, fácilmente reconocibles y cuya información descriptiva puede ser encontrada en la Web. Así que otro método de encontrar información asociada a las entidades ha sido la búsqueda en el grafo de conocimiento Knowledge Graph, que Google ofrece a través de su API.

El Knowledge Graph es una estructura de información enlazada por los conceptos semánticos —lo que usa Google para mostrar resultados adecuados a las necesidades de los usuarios tras una búsqueda. Este grafo contiene principalmente información procedente de fuentes abiertas, como es la Wikipedia, e incluye metadatos asociados a los conceptos que lo caracterizan en detalle.

Esta fuente de datos es realmente interesante para obtener información relativa a lugares, eventos o personas. El script de consulta al API hace distinción entre los tipos de resultados obtenidos. Los resultados también muestran el grado de confianza del éxito en la correspondencia, así que en el caso de encontrar una coincidencia del tipo esperado, registra los datos necesarios para la aplicación: descripción, título alternativo, URL de la entidad (página de la Wikipedia), URL de la imagen relacionada y licencia asociada, tipo de entidad normalizado usando schema.org (p.e., CivicStructure o LandmarksOrHistoricalBuildings). Por ejemplo, el resultado obtenido para la entidad [Plaza de] ‘EUROPA’, es:

Nombre: Europa

Score (confianza): 244.048874

URL: https://es.wikipedia.org/wiki/Europa

Tipo: Continent, Thing, Place, AdministrativeArea

Descripción: Europa es uno de los continentes que conforman el supercontinente euroasiático, situado entre los paralelos 36º y 70º de latitud norte.

Imagen: http://t0.gstatic.com/images?q=tbn:ANd9GcS0uZNMs4cfxsd-XAvy8iNVY-6CjX9gMCUV2BSloYIjgEQewznu

Licencia de la imagen: https://en.wikipedia.org/wiki/Wikipedia:Text_of_Creative_Commons_Attribution-ShareAlike_3.0_Unported_License

URI del Knowledge Graph: kg:/m/02j9z

Lamentablemente, Google ha quitado la resolución y visualización de las entidades del Knowledge Graph (hace unas semanas se podía visualizar un infobox con la información relativa a la entidad), pero no afecta a esta aplicación.

Paso 5: Tipos de Entidades

El Knowledge Graph, y sus consultas fueron la base para el comienzo de la clasificación de las entidades de manera automatizada. En el ejemplo anterior, Europa es de tipo: Thing (cosa, por supuesto); Continent (continente); Place (lugar); y AdministrativeArea (área administrativa).

Aproximadamente, un 40% de las entidades representadas fueron clasificadas de forma automática, siempre bajo la supervisión humana, ya que hay casos confusos en los que la inteligencia artificial no puede hacer nada.

Por ejemplo, la Calle ‘Cabrales’ se refiere al concepto ‘Cabranes’. Dentro del Knowledge Graph, ambos conceptos son identificados y descritos como municipios asturianos, pero realmente la calle es por una persona.

Una vez unidas todas las tablas (listado, geoposición, descripciones con el método Piñera y el Knowledge Graph, y tipos) se procedió a completar los tipos y posterior homogeneización semi-automática. Para ello se utilizó OpenRefine.

Para finalizar, se hizo un ejercicio de card sorting enfocado a agrupar conceptos por temáticas y definir una taxonomía intuitiva, que permita la clasificación de las entidades. Como resultado, las 1035 entidades obtenidas, se agrupan en:

(58) Artistas (músicos/as, pintores/as, actores/actrices, deportistas);

(53) Conceptos (sentimientos, oficios, otros elementos físicos)

(11) Exploradores (navegantes, conquistadores, etc.)

(220) Geografía (concejos asturianos, accidentes geográficos, lugares poblados, etc.)

(113) Historia (eventos históricos, personas, lugares y parajes de relevancia histórica)

(69) Industriales (personas relacionadas con el desarrollo industrial o urbanístico)

(112) De Letras (literatos, pensadores, filósofos, etc.)

(105) Naturaleza (fauna, flora y otros conceptos naturales)

(21) Nobles (monarcas y nobleza en general)

(27) Organizaciones (empresas o entidades organizativas)

(94) Polític@s (gobernantes, militares y administradores públicos)

(79) Religión (personas o lugares relacionadas con la religión)

(73) Científic@s (matemáticos, físicos o relacionadas con la ciencia o tecnología)

Asimismo, aprovechando los tipos más avanzados de los que se disponía, se estableció un segundo nivel en la taxonomía de temas. Por ejemplo, el tema Geografía incluye los subtemas: País (27 entidades), Geografía de Asturias (66), Geografía de España (44), Geografía de Gijón (21), Geografía Internacional (13), Lugares históricos (1), Orografía (48).

Puedes echarle un vistazo al resultado de Mis Calles Gijón/Xixón, jugar con ella y enviarme tus comentarios.

#miscalles #gijón #xixón #desarrollo #python #knowledge graph #kg

0 notes

espinr · 7 years ago

Photo

Cable Car Chasing (at San Francisco, California)

0 notes

espinr · 7 years ago

Text

Activ-IoTy Timekeeping - Report

Watch the video summarizing all the work done in this project in just 3 minutes.

Activ-IoTy Timekeeping is a system that the concepts of rapid prototyping and Internet of Things to develop a modular low-cost solution for timekeeping and competition management.

Description of the project

Athletics is perhaps the most popular sport in the world. It’s not only the most followed discipline during the Olympic Games, but Jogging is also extended in all countries as a base sport for kids or adult people. Fun races are also becoming popular, even among those who don’t do sports regularly. This trend is affecting positively to health and good habits of people.

More than 30 thousand running events take place in the U.S. every year, including 5k, 10k and 8K/5 mile races as well as half-marathons and marathons (source). It is estimated that there are over 47 million of runners (doing running/jogging or trail running) in the U.S.

In EU28, figures show that running is part of the way of life of many people. For instance, the 31% of the Danish population are runners, 25% in Germany (source). Most of these runners take part in competitions.

To make competitive running attractive among athletes, there are two important aspects to take into account: accurate/official timekeeping, and publication of comparable results for statistics and competition. We can achieve this with the complex and reliable systems of the competition organizers but this implies high costs. Also, these timing systems are not available in some developing countries.

Just as an example, in the following image of the Kenyan Cross Country Championships this year. There you can see how the fastest athletes in the world are selected to run in the World Championships, for instance. It shows an ordered queue of athletes, just after crossing the finish line. Two officials take notes on cardboard tables. These lists of results are published but, they won't be interoperable at all.

Winners of the Kenyan Cross Country Championship 2018

The solution

Activ-IoTy Timekeeping is a system that the concepts of rapid prototyping and Internet of Things to develop a modular low-cost solution for timekeeping and competition management. This system is based on standard technologies that enable plug and play different devices and share information with external servers to process the results, records, etc.

The following diagram shows the main components of Activ-IoTy: Checkpoints and Controller(s).

Checkpoints registers both competitor's ID and timestamp. These components are the MQTT publishers.

Controller(s) collects the information sent by the Checkpoints and processes the information. So, these are the MQTT subscribers.

Other third-party subscribers may perform other activities (i.e., visualizations, integration with third-party services, etc.)

Checkpoints

Those components perform the timekeeping activities. They are 'virtual gates' where competitors check-in to complete the full itinerary of the race. The objective of a checkpoint is to inform the system which competitor arrives at that specific location, timestamping that check-in. Checkpoints are multimodal and fully configurable. They can be designed and implemented with different IoT components to satisfy the requirements of the race and the organizers.

This pilot includes several checkpoints with different devices, platforms, and configurations. This serves to test a wide range of racing scenarios.

Checkpoint 1: RFID Reader + Raspberry Pi + Python

This Checkpoint is based on a IND903 UHF RFID Reader, connected to a Raspberry Pi (or a Intel Up Squared). The reader controller, as well as the checkpoint are coded in Python. The MQTT Client is implemented using the Paho Python module.

It is important to understand that each competitor taking part in a competition is identified by two unique IDs:

the bib number: alphanumeric string that follows the organizers standards (it could be an alias, surname or bib number). In this case, an unsigned integer; and

an EPC (Electronic Product Code) corresponding to the RFID tag attached to their bib number. This tag follows the EPC C1G2 (Class 1 Generation 2) standard.

Check all the details and code in Github page of this component.

The cost magnitude of this checkpoint depends on the antenna. The cost of the implementation of this pilot was around $150 (including RFID reader, sample tags, (5-6m reading range) antenna plus Rasberry Pi).

Checkpoint 2: Bluetooth Keypad + Raspberry Pi + Python

This is a cheap option of checkpoint implementation.

Athletics events are usually controlled by officials with different roles such as: judges, timekeepers and starters. Other competitions must rely on officials with other roles, even volunteers or marshals. This checkpoint enables officials and volunteers to register athletes at checkpoints using a simple physical keypad to type athlete's bib numbers.

Since all athletes must wear a visible bib identifier, this implementation is compatible with the existing rules of official competitions. Keypad may be substituted by a complete keyboard. Of course, the keypad could be a capacitive screen (i.e., tablet, smartphone, etc.) but it is probed that data input operators are more efficient with physical typing devices. Also they may be robust, even waterproof.

Check all the details and code in Github page of this component.

The cost of this checkpoint is below $50.

Checkpoint 3: IR Remote + Arduino Uno

Having in mind a less optimistic scenario –low budget, few resources–, I've implemented another checkpoint using a different solution. This option would cost less than $50 and it's really easy to use. It can be developed with a components from a basic Arduino-like kit.

A system based on Infrared (IR) remote control, is not very reliable for our purpose since there must be a direct visual line between transmitter and receiver –likely TV remote controls. More than a real solution, this is a proof of concept.

Check all the details and code in Github page of this component.

The cost of this checkpoint is also below $50.

MQTT publication

Checkpoints send an MQTT message once an athlete arrives. It is of the {checkpoint-ID}/checkpoint topic:

{ "checkpoint" : { "id" : "…", // Required "geo" : { "lat" : "…", "lng" : "…" } // Optional }, "bibId" : "…", // Either 'bib' or 'epc' are required "epc" : "…", "timestamp" : 0000000, // (NTP) UNIX time Required }

For check-in messages, it is required the checkpoint identifier (some information may be added, such as coordinates in case the checkpoint is on the move), the unique identifier of the competitor (we can have several, such as bib number and RFID tag), and the timestamp of the check-in.

Check more information about the PubSub design in the project.

Scalability and further options

Trying to expand the features and ecosystem of the platform, a mobile checkpoint was created with a Fona 3G module and an Arduino Lilypad.

Check more information about this checkpoint on the move.

Controller(s)

A Controller is basically a responsive Web application that enables competition management, including the processing of check-ins performed on checkpoints along the race course. There may be several controllers for different purposes, implementing different features. The controller of this pilot enables a complete management of the competition, from the registration and management of athletes, checkpoints, race, start list and visualizing the competition in real time.

Check the full video of a real example (at high speed) on YouTube.

Read more information about the Controller and its implementation. In case you want to test it, drop me an email and I'll send you the admin credentials. You could sign up the app, but with the role of athlete, so you won't be able to create and manage races (only admins can do it).

Some other components such as the MQTT Broker are described in a more detailed document about components of the project.

The strongest point of Activ-Ioty Timekeeping is scalability thanks to its design based on device independence and open standards.

The strongest point of Activ-Ioty Timekeeping is device independence so any kind of devices may be used to implement checkpoints. Likely checkpoints, controllers may be designed to manage the information to cover any competition requirement (e.g., relay races, multi-lap courses, time-trial races, and others).

Technology and Standards

The cornerstone of Activ-IoTy Timekeeping is the implementation based on open standards. Since the platform is designed to allow different and heterogeneous devices and implementations, standard technologies are crucial for scalability, enhancement of the system, and also to preserve the integrity of data.

Design and implementation were focused on the use of standard technologies, open source libraries and mounted on open hardware devices. Python v3 and Embed C++ are the selected programming languages for the devices (Checkpoints); JavaScript, for the Controller.

Obviously, everything will run on the Internet. There are various underlying protocols since Activ-IoTy Timekeeping aims to be flexible in terms of physical connectivity. Controllers and Checkpoints may be installed in locations with strong WiFi signal (city or, town center), even with access to wired Ethernet routers, but also in the middle of the forest where we would need establish cellular GSM or 3G connections. Thus, the physical layer includes: IEEE 802.3 (Ethernet), IEEE 802.11 (WiFi) and WCDMA/HSDPA (3G) protocols.

In this sense, most of the tests were performed on WiFi access points, but the Mobile Checkpoint used a 3G cellular connection.

On top of the Internet, the transport layer is composed mainly by TCP or Transmission Control Protocol. UDP or User Datagram Protocol is used for specific applications such as time synchronization through external NTP (Network Time Protocol) servers.

MQTT, key in communications

Checkpoints, controllers and the rest of the modules are based on the Pub/Sub (Publication/Subscription) paradigm. Since the distributed system needs reliability and there is a potential lack of quality of communications, the solution uses MQTT (Message Queuing Telemetry Transport) as messaging protocol. MQTT allows Checkpoints (publishers) to send messages to the Controllers (subscribers). A broker, up and running in a reliable environment, will be in charge of delivering those messages effectively.

MQTT implements different levels of Quality of service (QoS). Integrity of data and rapidity is key for Activ-IoTy. Also intermediate Checkpoints may be weak in terms of connectivity. Thus the system will implement QoS 1 (at least once). Using QoS level 1, it is guaranteed that a message will be delivered at least once to the receiver. Messages can be delivered more than once but this will be taken into account and solved by the Controller.

The MQTT broker is implemented and running on a reliable UP-Squared accessible through the port 1883 of activioty.ddns.net. Regarding security, this broker does not provide a secure channel. This is a clear breach in the system but was done on purpose to offer real-time demonstrations in this prototype (i.e. jury of the challenge could subscribe to the MQTT queue in order to check the real messages running on the system, like in the example). In case of real operation, the broker would implement either user authentication and SSL-secured transmissions.

UpSquared Board used for prototyping and where Acti-IoTy Timekeeping servers run

Web Standards

The system implements some examples of visualizations and operation on the system. The Controller manages a MongoDB where stores all the information produced by the system, including the MQTT messages about competitors' check-ins at checkpoints. Additional Web services may be exposed on top of that database, giving access to third parties to build additional services or products. Just as an example of this, the system may offer HTTP (Hypertext Transfer Protocol) RESTful services to get race results. Value of data is clear, so this may increase the usefulness of the platform, enabling innovation in further developments (e.g., integration with social networks, better visualizations, etc.).

The final design was complemented by other specific top level standards that guarantee interoperability among similar systems around the world. The description of underlying entities of the database using the OpenTrack vocabularies, an initiative to model the domain of Athletics in a standard way. Through this Schema.org based vocabulary (and taxonomies), the system describes athletes, clubs, competitions and results in a semantic way. Check the results generated after the test in JSON-LD format.

Applicability to industry

As mentioned before, the running/jogging market is growing every year and competitions happen everywhere at any time. There are many applications and products that probe the importance of competitiveness in this sport. For instance, most of the sports tracking mobile apps (Strava, Nike+, Endomondo, etc.) include gamification based on performances and rankings. So, Activ-IoTy is part of this trend.

It is important to say that reliable timekeeping systems already exist. Indeed, this is problem solved since years ago. We are used to seeing real-time results while we watch Athletics at the Olympic Games on TV. In this case, a few big companies provide timing services to Athletics Federations to cover these competitions. Those costs are not affordable for the majority of race organizers. Another problem is the lack of open standards in those solutions, so most of them are proprietary and closed.

Activ-IoTy Timekeeping brings innovation, new business opportunities and social benefits

Because of this, Activ-IoTy takes advantage of this running bubble, enabling an open platform to enrich sports. As part of the open concept (i.e., open source, open hardware, open data, open standards), Activ-IoTy enables SMEs to develop new business opportunities, offering timekeeping services, including new devices and mechanisms, enhancing the final products, fostering innovation.

As shown in the tests, this pilot is fully functional, but not very robust –only basic features covered, and lack of testing in depth. It could become a robust platform where build services on top of it (i.e., consultancy services, specific applications, new check-in methods, integration with third-party services, etc.). Apart from the platform, there is a clear value of the data produced. Information managed by Activ-IoT could be also part of the open data ecosystem, so this brings innovation, new business opportunities, and benefits for the community.

As a next step, apart from evolving the hardware of the platform, the controller should implement APIs to communicate with the current commercial sports systems. Integration with other platforms is key to a quick success of the platform.

Lessons learned

From software development to hardware prototyping. Back in December, I oriented the design phase as a software project –I have some experience in this field. When I started prototyping the checkpoints I found many issues that drove me crazy. I expected some parts to be easier, from the basic configuration of a device (e.g., installing the right BIOS, operating system, and drivers for the Intel UpSquared took me a few hours before it worked perfectly), to understanding why some implementations didn't work as expected (e.g., feeding the RFID reader with lower amperage than recommended). So, for this kind of projects, one have to take into account more variables than a simple project of software development.

Community is key to the growth of IoT. Developing new IoT solutions is a piece of cake. The Web is full of IoT enthusiasts aiming to help. I faced many issues during the implementation phase that were solved in minutes thanks to forums and blog posts on the Web. So, give attribution and share as well!! Eating my own dog food, all, code and documentation, is available in the Activ-IoTy Github Repository.

Debugging is so hard… Most of the developments I've made for this project were based on existing libraries. Almost all worked properly at the first try, but not all. In case of low-level programming languages, such as some Arduino sketches, debugging was difficult in some parts. The lesson here is… be patient:-)

Python for arrays and bytes management. Other software I used, such as the RDID-reader code, was created from scratch. Although I had no experience with it, I chose Python for its expected support of arrays and hexadecimal bytes required for the low-level communications protocol. It was a good experience. It's an intuitive programming language, with infinite modules to solve any challenge you face. The code may be run on some of the devices I had for the pilot (i.e., UpSquared and Raspberry Pi).

Beauty !== functional. I bought an RFID Reader that has worked right from the first try. I used an 8dBi ceramic antenna which was enough to read tags properly in a 5-meter range. I decided to pack everything to make it portable and nice, so I designed and printed a case for a Raspberry Pi, the reader and the antenna. Making it from the scratch would be difficult, but I used an existing 3D model for a Raspberry Case on TikerCad to [create it](https://github.com/espinr/activioty/tree/master/docs/checkpoints/rfid-reader-python/3D_model_RFID Reader + Antenna + RPi.stl).

Plastic shouldn't shield too much the readings so I designed the protective case with the antenna on top of the rest of the components.

I was proud of the professional result… Until I tested it :-) All devices worked properly, but RFID reading range decreased to a few inches. It was used to do some tests in the field (as seen in the video), but that time, the system missed around the 75% of the tags.

I learned that both antenna and RFID tag positioning is crucial for this. Experts use the acronym SOAP to refer the four main aspects of tag positioning: Size, Orientation, Angle, and Placement. So, the recommendation is test it (again and again), debugging the Receive Signal Strength Indication (RSSI) when a tag was read.

Mosquitto and Paho are cool. I had never worked with queues. I chose MQTT and it was a really good idea. Those MQTT implementations are really stable, lightweight and easy to configure and use.

Haste makes waste. Please, forgive me for the ugly name I chose for this project. It was a real improvisation. I sent the proposal as soon as I heard of the Challenge, but this was the last day, after work (just finished the Open Source for Industry 4.0 Webinar). So I improvised and submitted a vague idea I had in mind without caring too much. Now I regret it :-)

It was hard but I enjoyed a lot. Nothing more to say :-) I think this was my first step in the IoT world. At least, I'll keep playing with Activ-IoTy Timekeeping. Yes, I know, In case I start some business around it, I'll change the name.

#activioty #activioty openiotchallenge #iot #eclipse #mqtt #standards #Timekeeping #running #race #openiotchallenge #report

0 notes

espinr · 7 years ago

Text

Checkpoints on the Move?

Competitors Tracking for Activ-IoTy

Update of the ActivIoTy project for the Eclipse Open IoT Challenge.

We are on the last mile of the challenge, and I’m exploring new ways to enrich the user experience of the platform, adding new features. The system is based on fixed checkpoints that informs organisers and the public about the status of the competition. In between of these points there is no information. The general public standing at the finish line of a competition is interested in having a constant feedback. So the system should offer information in real time about certain competitors —for instance, the first male and female in the competition.

Tracking the position of an athlete is really simple nowadays. Indeed, you can do it with mobile apps such as Strava. Carrying the smartphone is not a solution in this case, due to competitors try to avoid unnecessary overweights. Also, in case of runners that want to use this solution, in rainy days and on muddy trail courses is not a good idea. Also in ultra-distance races is not feasible due to battery lifetime.

The solution I propose is to create an active transponder that broadcast real-time notifications with the GPS location of the competitor on a GPRS connection. The aim of this component is informing in real time about the status of the race. It will be attached (e.g., sewed to a vest) to the official or volunteer that leads the race (on car, or bike).

For this, I chose the Adafruit FONA 3G Cellular Module. The European Version of this module provides WCDMA + HSDPA support, with GSM backwards-compatibility, and a built-in GPS module for geolocation. It was controlled by a lightweight Arduino Lilypad board, and powered by a 3.7v LiPo Battery.

Assembly is really easy with this Adafruit tutorial. This tutorial also offers a library and examples for Arduino. The implementation for my Lilypad was easy to adapt, just power connections and configuration of RX, TX and RTSin (flow control). If you use another Arduino model, and other pin configuration, you'll have to check its interrupt pins that are required for TX/RX.

I've experienced problems with the code, specially with the responses to the FONA3G commands. After introducing several delays in the execution of the sketch, it runs the basic tasks we need:

Connect to the module and detect the version (European Version);

Identify the SIM IMEI;

Detect and connect the network;

Configure (APN, username and password) and establish a GPRS connection (for a Vodafone Spain SIM card);

Connect and fix the GPS;

Read the GPS location and submit coordinates to a web server;

(Repeat from 5)

The location collected will be sent to a script that stores and processes the information. Since the purpose of this is just testing, those coordinates will be forwarded to a variable called 'First Competitor' within Ubidots. I chose this platform because it provides an easy way to represent lat/log tracking of devices.

This is the main code of the sketch I used (trimmed for better legibility):

#include "Adafruit_FONA.h" #define FONA_RX 2 #define FONA_TX 3 #define FONA_RST 5 #include <softwareserial.h> SoftwareSerial fonaSS = SoftwareSerial(FONA_TX, FONA_RX); SoftwareSerial *fonaSerial = &fonaSS; Adafruit_FONA_3G fona = Adafruit_FONA_3G(FONA_RST); void setup() { while (!Serial); Serial.begin(115200); Serial.println(F("Initializing FONA 3G...")); fonaSerial->begin(4800); if (! fona.begin(*fonaSerial)) { Serial.println(F("Couldn't find FONA")); while (1); } type = fona.type(); Serial.println(F("FONA is OK")); // Print module IMEI number. char imei[16] = {0}; uint8_t imeiLen = fona.getIMEI(imei); if (imeiLen > 0) { Serial.print("Module IMEI: "); Serial.println(imei); } fona.setGPRSNetworkSettings(F("airtelwap.es"), F("wap@wap"), F("wap125")); fona.setHTTPSRedirect(true); printBatteryLevel(); waitForNetworkConnection(); delay(2000); printRSSI(); delay(2000); turnGPRSOn(); delay(5000); turnGPSOn(); delay(5000); while (!isGPRSEnabled()) { delay(2000); } } void turnGPRSOn() { Serial.println(F("Enabling GPRS...")); fona.enableGPRS(true); } void turnGPSOn() { Serial.println(F("Enabling GPS...")); fona.enableGPS(true); } uint16_t isGPRSEnabled() { uint16_t status = fona.GPRSstate(); Serial.println(F("GPRS status:")); Serial.println(status); return status; } void waitForNetworkConnection() { // read the network/cellular status uint8_t n = 0; while (n!=1) { Serial.println(F("Checking network connection")); n = fona.getNetworkStatus(); delay(500); } Serial.println(F("Registered (home)")); } void printBatteryLevel() { uint16_t vbat; while (! fona.getBattPercent(&vbat)) { Serial.println(F("Failed to read Battery")); } Serial.print(F("Battery level = ")); Serial.print(vbat); Serial.println(F("%")); } void updateValues(float lat, float lng) { // read website URL uint16_t statuscode; int16_t length; String latitudeStr = String(lat); String longitudeStr = String(lng); String url = String("activioty.ddns.net/tracker/update?bibId=1&lat=" + latitudeStr + "&lng=" + longitudeStr); Serial.println(F("GET...")); Serial.println(url); Serial.println(F("****")); if (!fona.HTTP_GET_start(url.c_str(), &statuscode, (uint16_t *)&length)) { Serial.println("Failed!"); return; } while (length > 0) { while (fona.available()) { char c = fona.read(); Serial.write(c); length--; if (! length) return; } } Serial.println(F("\n****")); fona.HTTP_GET_end(); } void gpsFix() { // check GPS fix fona.GPSstatus(); } void checkGPSLocation() { char gpsdata[120]; fona.getGPS(0, gpsdata, 120); Serial.println(F("Reply in format: mode,fixstatus,utctime(yyyymmddHHMMSS),latitude,longitude,altitude,speed,course,fixmode,reserved1,HDOP,PDOP,VDOP,reserved2,view_satellites,used_satellites,reserved3,C/N0max,HPA,VPA")); Serial.println(gpsdata); } void loop() { printBatteryLevel(); Serial.println(F("GPS Status...")); gpsFix(); delay(5000); Serial.println(F("GPS Location...")); float latitude, longitude, speed_kph, heading, altitude; // if you ask for an altitude reading, getGPS will return false if there isn't a 3D fix boolean gps_success = fona.getGPS(&latitude, &longitude, &speed_kph, &heading, &altitude); delay(1000); if (gps_success) { Serial.print("GPS lat:"); Serial.println(latitude, 6); Serial.print("GPS long:"); Serial.println(longitude, 6); Serial.print("GPS speed KPH:"); Serial.println(speed_kph); Serial.print("GPS heading:"); Serial.println(heading); Serial.print("GPS altitude:"); Serial.println(altitude); Serial.println(F("Updating values...")); updateValues(latitude, longitude); } else { Serial.println("Waiting for FONA GPS 3D fix..."); } delay(30000); }

You can check the full code on Github.

Yes, you realized of something weird up there. The updateValues function makes a HTTP GET call to update values. I have some problems to use the POST method with that FONA library. Anyway, for testing purposes I created a script that posts the information from activioty.ddns.net to Ubidots.com.

I keep experimenting with this module.</softwareserial.h>

#activioty openiotchallenge #Openiotchallenge #iot #internet of things #fona3g #arduino #lilypad #tracking #gps

0 notes

espinr · 7 years ago

Text

The (Activ-IoTy) Controller

Update of the ActivIoTy project for the Eclipse Open IoT Challenge.

Activ-IoTy is composed of Checkpoints (devices that collects competitors' IDs, time-stamps it, and publish it), the MQTT Broker (an instance of Mosquitto Server) and the Controller. The Controller is the component that performs most of the platform's logic.

The Controller is a MQTT subscriber that processes the checkins performed on Checkpoints.

The Controller is basically a Web application that enables competition management, including the processing of checkins performed on checkpoints along the race course. There may be several controllers for different purposes, implementing different features. This controller allow us performing the following tasks:

Features

Athletes Management

Users that can sign-up into the platform and submit their profiles.

Checkpoints Management

Creation/edition of checkpoints. This feature allows describing checkpoints that can be located on a map. Checkpoints are identified by their ID.

Race Definition

Creation/edition of races. We can describe races, including a geo-JSON file with the course and the **list of Checkpoints ** that athletes must visit to complete the race. Checkpoints might require more than a 'checkin' to complete the race (i.e., multi-lap races). This example is limited to just 1 lap.

Obviously, only one Checkpoint can be at the finish line (it has to be selected using a radio button).

Start List Creation

Registered athletes may be included as competitors in specific races. There is an automatic process where selected athletes are included as competitors in a race. Competitors have assigned unique bib identifiers.

Bib identifiers must be linked to RFID tags. For this example, I use this map of unique identifiers (included in the collection identifiers):

[ { bibId: '001', epc: 'E2 00 51 42 05 0F 02 62 16 00 6F 40' }, { bibId: '002', epc: '00 00 00 00 00 00 00 00 00 00 03 11' }, { bibId: '003', epc: '00 00 00 00 00 00 00 00 00 02 49 17' }, { bibId: '004', epc: 'E2 00 51 79 98 18 01 38 02 50 F4 9E' }, { bibId: '005', epc: 'E2 00 51 86 06 0A 01 53 08 00 C3 10' }, { bibId: '006', epc: 'E2 00 51 79 98 18 01 38 01 50 F9 BB' }, { bibId: '007', epc: 'E2 00 30 16 66 13 01 21 15 50 74 EF' }, { bibId: '008', epc: 'E2 00 41 06 21 03 00 32 10 10 AF 41' }, ]

For this prototype, I've populated the database, loading a JSON file directly.

The following picture shows the start list of a race. All athletes shown are the users registered in the system. The administrator may assign bib numbers to athletes directly.

Real-time Competition

Once a race was defined, including the checkpoints assigned to the course, and the start list already created, the competition is almost ready to start. The admin is able to open the 'live results' option. In this screen, a map with the course track and the checkpoints are shown.

At the beginning, the markers representing checkpoints are in red (disabled). Now, the controller is waiting for the ready MQTT messages from checkpoints. As soon as a checkpoint is ready, it sends a {checkpoint-ID}/ready message like this (is the first message the controller received during the example):

TOPIC: Keyboard-2/ready

MESSAGE:

{ "checkpoint" : { "id" : "Keyboard-2" }, "timestamp" : 1519401196 }

Checkpoints are marked in white and their infowindow is shown.

At the right-hand side of the image you can see a terminal connected to the same MQTT broker, waiting for the +/ready messages that are processed by the controller.

The competition starts (the admin presses the 'play' button'). Once the stopwatch is running the controller expects +/checkin messages from the checkpoints along the course. The following GIF shows the moment when competitors arrived at the first checkpoint (ID: Keypad-1).

The first checkpoint is powered by a keypad so the device sends the bib number of the athlete in a message like the following example (it is the same the checkpoint sent when the first athlete did the checkin). In the right-hand side of the video, you can see a terminal with a Mosquitto client waiting for the same +/checkin messages as the controller does).

TOPIC: Keyboard-2/checkin

MESSAGE:

{ "checkpoint" : { "id" : "Keyboard-2" }, "bibID" : "002", "timestamp" : 1519401513 }

The competition may be followed in real time as shown in the full video –the video was speeded up to show just the relevant events: readiness of checkpoints and checkins at all checkpoints. On the right hand side, there are two terminals running Mosquitto subscribers to see the real messages processed by the controller.

Complete video:

[ Check the full video at http://www.youtube.com/watch?v=nTlSV7WbGoE

Results in Open Format

Final results may be exported once the admin considers that the race is finished (other intermediate results could be exported).

This data is served in JSON-LD, following the OpenTrack model. This vocabulary is based on schema.org and is in process of standardisation.

Check the results generated in this example –perhaps the final IRIs does not resolve, it's just an example.

Implementation

The Controller is developed in Meteor, an open source platform for Web, mobile, and desktop applications build in JavaScript. The resulting mobile web application is compatible with all the latest browsers and it has been designed using the responsive web design approach, in concrete through the Materialize CSS framework.

In terms of technology, the application is built in JavaScript and is run on a Node.js server, and a MongoDB as the subjacent database system.

The application is fully operational and you can test it, but it is in an early development phase so it implements exactly the requirements of the pilot.

MQTT for in Javascript

The system uses MQTT.js, a client library for the MQTT protocol. It is written in JavaScript for node.js and any browser. It can be installed through NPM with just the following command:

npm install mqtt --save

Since the application must be subscribed to the MQTT queue, mqtt.js is configured in the server side of Meteor to wait for new messages from checkpoints. During the startup of the Controller, the a new subscription is setup (to topics +/ready and +/checkin. Also the callbacks of these events are configured.

Once a new checkin is detected, the Controller creates a new document on the checkins collection in the database with all the information of the received checkin. Also, when it detects that a checkpoint is ready, a ready flag in the database is updated to true. The concrete code is this:

/* * File at /imports/startup/server/mqtt.js */ import { Meteor } from 'meteor/meteor'; import mqtt from 'mqtt'; import Sntp from 'sntp'; import { Checkpoints } from '../../api/checkpoints/checkpoints.js'; import { Identifiers } from '../../api/identifiers/identifiers.js'; import { Checkins } from '../../api/checkins/checkins.js'; if (Meteor.isServer) { const mqttBroker = { url : 'mqtt://activioty.ddns.net', port: 1883, }; const mqttClient = mqtt.connect(mqttBroker.url, mqttBroker.port); mqttClient.on('connect', () => { const timestamp = Sntp.now(); mqttClient.subscribe({ '+/ready': 1, '+/checkin': 1 }); mqttClient.publish('controller/ready', `{ "checkpoint" : { "id" : "I am the controller :-)"}, "timestamp" : ${timestamp} }`); }); mqttClient.on('message', Meteor.bindEnvironment((topic, message) => { const timestamp = Sntp.now(); console.log(`MQTT Message received [${topic.toString()}] -> ${message.toString()}`); if (topic.includes('/ready')) { // Marks the checkpoint as ready try { const msg = JSON.parse(message.toString()); Checkpoints.update({ identifier: msg.checkpoint.id }, { $set : { ready : true } }, (error, result) => { if (error) { console.log(error); } if (result) { console.log(`${result} Checkpoint ${msg.checkpoint.id} is marked as ready (${msg.timestamp})`); } }); } catch (e) { console.log('I cannot process the readiness of a checkpoint, so I skip it'); } } else if (topic.includes('/checkin')) { // Includes the athlete in the checkins database try { const msg = JSON.parse(message.toString()); const checkpoint = Checkpoints.findOne({ identifier: msg.checkpoint.id }); let identifier; if (msg.epc) { identifier = Identifiers.findOne({ epc: msg.epc }); } else { identifier = Identifiers.findOne({ bibId: msg.bibId }); } const docToInsert = { checkpointId: checkpoint._id, bibIdentifier: identifier.bibId, epc: identifier.epc, timestamp: msg.timestamp, }; Checkins.insert(docToInsert, (error, result) => { if (error) { console.log(error); } if (result) { console.log(`Checkin of Bib Number ${identifier.bibId} at ${checkpoint.name}`); } }); } catch (e) { console.log('I cannot process the checkin in a checkpoint, so I skip it'); } } })); }

The controller will process and work directly with the checkins collection in the database.

mqtt-collection package for Meteor

Another alternative, that was tested but not used in the final solution is mqtt-collection, a MQTT package for Meteor that collects and stores the MQTT messages received. Messages are stored in the MongoDB directly, enabling reactive support for the client side.

This package is easy to use. The following code includes the definition of the MQTT messages collection and establishes the connection with the MQTT broker, indicating the topics.

/* * File at /imports/api/messages/messages.js */ import { Meteor } from 'meteor/meteor'; import { Mongo } from 'meteor/mongo'; export const Messages = new Mongo.Collection('messages'); if (Meteor.isServer) { Messages.mqttConnect('mqtt://activioty.ddns.net:1883', ['+/ready', '+/checkin'], { insert: true }, {}); }

Timing

In order to adjust time to the common NTP servers, the system uses the sntp package. This is a SNTP v4 client (RFC4330) for Node.js. It allows the server connecting to the NTP server and returns the server time along with the roundtrip duration and clock offset. To adjust the local time to the NTP time we would need to add the returned offset to the local time.

The main stopwatch of the application is built with aldeed:clock, An accurate reactive clock package for Meteor that retains elapsed time. Usage is simple and reliable.

This is a summary of the main declarations related to the official stopwatch of the competition:

/* * file at: /imports/ui/components/competition/competition.js */ Template.competitionMain.onCreated(() => { // ... template.CompetitionClock = new ReactiveClock('ExerciseClock'); template.CompetitionClock.setElapsedSeconds(0); // ... } Template.competitionMain.helpers({ stopwatch() { const template = Template.instance(); return template.CompetitionClock.elapsedTime({ format: '00:00:00' }); }, // ... }); Template.competitionMain.events({ 'click #startCompetition-button'(event) { const template = Template.instance(); template.CompetitionClock.start(); template.startTimestamp.set(Sntp.now() / 1000); // ... }, 'click #finishCompetition-button'(event) { const template = Template.instance(); template.CompetitionClock.stop(); template.finishTimestamp = Sntp.now(); // ... }, });

Athlete's Checkins

If the clock is enabled (ongoing competition), the controller system waits for checkin messages. The application runs this code, once a new checkin is included in the database:

Tracker.autorun(() => { // Gets the current Race template.currentRace = Races.findOne({ _id: raceId }); if (template.currentRace && template.currentRace.checkpoints) { template.currentRace.checkpoints.forEach((checkpoint) => { // creates a local array with checkpoints template.checkpoints.push(Checkpoints.findOne({ _id: checkpoint.id })); }); } // If the clock is running if (template.startTimestamp.get() && template.startTimestamp.get() > 0) { template.subscribe('checkins.after', template.startTimestamp.get()); template.checkpoints.forEach((checkpoint) => { const checkinsForCheckpoint = Checkins.find({ checkpointId: checkpoint._id }); if (checkinsForCheckpoint) { checkinsForCheckpoint.forEach((checkin) => { let competitor; if (checkin.epc) { competitor = Competitors.findOne({ epc: checkin.epc }); } else { competitor = Competitors.findOne({ bibId: checkin.bibId }); } const totalTime = checkin.timestamp - template.startTimestamp.get(); // The rest of the logic in the application (show it on a map, store results, etc.) checkinAthlete(template, checkpoint, competitor, totalTime); }); } }); } });

This pilot illustrates how to the data can be processed and visualized. Other services and products on top of it are unlimited.

Deployment and testing

Activ-IoTy Controller is deployed on a UPsquared I have at home.

We have done several tests in the real World.

I'll post the URL of the controller for you to test it if you want.

#iot #activioty openiotchallenge #activioty #mqtt #eclipse #paho #ntp #meteor #node.js #mongodb #timekeeping #athletics

0 notes

espinr · 7 years ago

Text

Implementation of Low-Cost Activ-IoTy Checkpoint in Python (Bluetooth Keypad)

Update of the ActivIoTy project for the Eclipse Open IoT Challenge.

This is a cheap option to implement an Activ-IoTy compatible checkpoint.

Athletics events are usually controlled by officials with different roles such as: judges, timekeepers and starters. Other competitions must rely on officials with other roles, even volunteers or marshals. This solution enables officials and volunteers to register athletes at checkpoints using a simple physical keypad to type athlete's bib numbers.

This solution only needs a bluetooth input device connected to a Raspberry Pi, or similar.

The official located nearby the checkpoint type the bib number and press enter to register athletes' checkinw. The system will send the information (bib number + timestamp) of this checkin automatically.

The implementation is based on the module I developed to implement the Checkpoint based on a RFID Reader.

Both approaches implement a generic Reader that performs two main tasks: initialization (setup of the checkpoint) and doInventory.

class Reader(object): def __init__(self): pass def initialize(self): pass def doInventory(self): pass

They also use a common class Checkpoint that implements all the basic functionality of a checkpoint:

Get reliable timestamps from a common NTP server;

Establish a connection with the MQTT server (once it is connected and set-up sends a {checkpointId}/ready message);

Through a Reader, read either bib numbers or RFID EPCs that are related to each competitor (once they are read, they are sent to the MQTT queue under the topic {checkpointId}/checkin);

This is executed in an infinite loop. The controller will be in charge of deal with the messages and do the business logic.

The Checkpoint is implemented using ntplib and paho.mqtt. This is a summary of the code:

# ... imports ... import ntplib import paho.mqtt.publish as publish class Checkpoint(object): TOPIC_READY = 'ready' TOPIC_CHECKIN = 'checkin' def __init__(self, id, reader, mqttBrokerHost, mqttBrokerPort=1883): ''' :param id: String with the identifier of the Checkpoint :param reader: Ind903Reader object (or other Reader) that will do the inventory :param mqttBrokerHost: String with the hostname of the MQTT broker :param mqttBrokerPort: (int) with the port of the MQTT broker (1883 by default) ''' self.id = id self.reader = reader self.mqttBrokerHost = mqttBrokerHost self.mqttBrokerPort = mqttBrokerPort self.timestampOffset = self.getOffsetNTPTime() def checkinCompetitor(self, idCompetitorEPC, idCompetitorBibNumber): ''' Method to be called once a competitor do a check-in. This is a callback method to be send to the reader. It sends a MQTT message to the broker :param idCompetitor: String with the ID of the competitor that the reader detected. ''' competitor = {} if (idCompetitorEPC != None): competitor["epc"] = idCompetitorEPC; if (idCompetitorBibNumber != None): competitor["bib"] = idCompetitorBibNumber; messageCheckin = { "checkpoint" : self.id , "timestamp" : self.getTimestamp(), "competitor" : competitor } topic = self.id + "/" + self.TOPIC_CHECKIN publish.single(topic, json.dumps(messageCheckin), hostname=self.mqttBrokerHost) print(topic + " topic to MQTT:") print(json.dumps(messageCheckin)) def execute(self): messageReady = { "checkpoint" : self.id , "timestamp" : self.getTimestamp() } topic = self.id + "/" + self.TOPIC_READY publish.single(topic, json.dumps(messageReady), hostname=self.mqttBrokerHost) print(topic + " topic to MQTT:") print(json.dumps(messageReady)) self.reader.initialize() self.reader.doInventory(self.checkinCompetitor) def getOffsetNTPTime(self): ''' Get timestamp from a NTP server and stores the offset with the internal clock. :return: (float) with the difference between the local clock and the NTP time (in seconds) ''' c = ntplib.NTPClient() response = c.request('europe.pool.ntp.org', version=3) # convert from seconds since 1900 to seconds since 1970 ntpUnixtime = response.tx_timestamp - 2208988800 timestampOffset = time.time() - ntpUnixtime; return timestampOffset def getTimestamp(self): ''' Returns the local time plus the offset registered at the beginning. :return: long with the current timestamp (unix time) ''' return int(time.time() + self.timestampOffset)

The key part of this code is the execution itself:

def execute(self): ... self.reader.doInventory(self.checkinCompetitor)

When the checkpoint is correctly setup, the reader is asked to start the inventory of runners. Once either a bibNumber (from a keyboard input) or a EPC (from the RFID reader) is detected the checkinCompetitor callback is executed. This method creates the message for the MQTT receivers that will contains the information about the competitor id.

The functionality of this specific checkpoint is implemented using the following main method:

# ... imports if __name__ == '__main__': reader = usb_keypad_reader.USBKeypadReader() checkpoint = checkpoint.Checkpoint('Keypad-1', reader, 'activioty.ddns.net') checkpoint.execute()

This approach may be extended to other checkin mechanisms such as bar-code readers or computer vision analyzers. Perhaps for the next release ;-)

#paho #eclipse #activioty openiotchallenge #openiotchallenge #rpi #raspberrypi #python #code #checkpoint #timekeeping #athletics #races

0 notes

espinr · 8 years ago

Text

IR Keypad on Arduino

Update of the ActivIoTy project for the Eclipse Open IoT Challenge.

This checkpoint would help a person from the staff (volunteer, official, marshal, whoever) register the bib number of competitors at specific points along the race course. Bib numbers will be introduced through a numeric keypad. The official would type the bib number and press enter. The system will send the information (bib number + timestamp) of this checkin automatically.

This module uses typical components available in most of the basic prototyping kits. So this could be implemented easily by anyone. In this case I've used components I bought years ago in my first Arduino kit:

Arduino One rev 3

CC3000 WiFi Shield. This could be changed for an Ethernet or 3G breakout, depending on the type of connection we will have).

SFH5110 Infrared Receiver. (Other models such as the TSOP312 are also welcome)

Generic remote IR control. (I used a generic IR remote control, but you can recycle your old TV remote control).

2 × LEDs. To provide feedback when a key is pressed and a bib number is sent (or delete the current one if mistaken).

2 × 220 ohms resistors.

The checkpoint would work in the same way as the rest of checkpoints:

Connect to the Internet (using the WiFi shield or any other method you need)

Get the official time from a NTP server in Unix time format.

In this case, the Arduino stores the offset between the internal clock and the NTP time. So it will calculate the synchronized time using the internal clock and the offset.

Send a MQTT ready message to the MQTT broker indicating the timestamp and the own ID.

Enter in a loop waiting for entries.

In this case, the Arduino enters in an infinite loop to read bib numbers read more.

The script will enter in a loop waiting for keys pressed (numbers). Once a 'number' is detected, it is included in a buffer. Two special keys will control when the bib number must be sent or deleted from the buffer. Keys in the keypad and their functions are configurable in the sketch.

IR Receiver and IRLIB2

The transmitter in the remote control sends out a stream of pulses of infrared light (humans cannot see) when a button is pressed. A transmitter is often a light emitting diode (LED) which is built into the pointing end of the remote control handset. The infrared light pulses form a pattern unique to that button.

The Arduino sketch uses the IRLib2 library to perform all the management of the IR receiver. If you are interested in in-depth information about this, you can refer to this tutorial from Adafruit, explaining all details about IR and how to use commercial remotes within your sketch.

I used the SFH5110 IR receiver directly connected to the Arduino's digital pin #2. You have to take into account that several digital pins are used by the CC3000 WiFi shield.

The sketch enters in a loop waiting for the receiver to read a signal (number or control key).

The receiver will read a unique 32-bit hexadecimal data that represents the key pressed by the user. If the receiver finds something weird (e.g., several buttons pressed at the same time), returns the 0 value. The interesting thing is we can use any remote control, we can map the keys we need and run the script.

The sketch used:

#include "IRLibAll.h" // LEDs connected to A0 and A1 analog pins // the rest of the digital pins are used by CC3000 const int LED_GREEN = A1; const int LED_RED = A0; // IR receiver configuration pin const int IR_RECEIVER = 2; //Create a receiver object to listen on pin 2 IRrecvPCI myReceiver(IR_RECEIVER); //Create a decoder object IRdecode myDecoder; // Key map (specific for the remote control) const uint32_t KEY_1 = 0xFF30CF; const uint32_t KEY_2 = 0xFF18E7; const uint32_t KEY_3 = 0xFF7A85; const uint32_t KEY_4 = 0xFF10EF; const uint32_t KEY_5 = 0xFF38C7; const uint32_t KEY_6 = 0xFF5AA5; const uint32_t KEY_7 = 0xFF42BD; const uint32_t KEY_8 = 0xFF4AB5; const uint32_t KEY_9 = 0xFF52AD; const uint32_t KEY_0 = 0xFF6897; const uint32_t KEY_OK = 0xFFC23D; // The green button (play) const uint32_t KEY_CLEAR = 0xFF22DD; // The blue 'prev' button (clear current input) void setup() { Serial.begin(115200); delay(1000); while (!Serial); pinMode(LED_GREEN, OUTPUT); pinMode(LED_RED, OUTPUT); myReceiver.enableIRIn(); Serial.println(F("[ActivIoTy] IR reader started")); } void blinkLED(int led, int delayms) { digitalWrite(led, HIGH); // turn the LED on delay(delayms); // wait for delayms digitalWrite(led, LOW); // turn the LED off } void loop() { String bibNumber = ""; // Bib number or name in it bool bibComplete = false; while (!bibComplete) { //Continue looping until you get a complete signal received if (myReceiver.getResults()) { myDecoder.decode(); //Decode signal received long value = myDecoder.value; switch (value) { case KEY_0: bibNumber = String(bibNumber + "0"); blinkLED(LED_GREEN, 100); break; case KEY_1: bibNumber = String(bibNumber + "1"); blinkLED(LED_GREEN, 100); break; case KEY_2: bibNumber = String(bibNumber + "2"); blinkLED(LED_GREEN, 100); break; case KEY_3: bibNumber = String(bibNumber + "3"); blinkLED(LED_GREEN, 100); break; case KEY_4: bibNumber = String(bibNumber + "4"); blinkLED(LED_GREEN, 100); break; case KEY_5: bibNumber = String(bibNumber + "5"); blinkLED(LED_GREEN, 100); break; case KEY_6: bibNumber = String(bibNumber + "6"); blinkLED(LED_GREEN, 100); break; case KEY_7: bibNumber = String(bibNumber + "7"); blinkLED(LED_GREEN, 100); break; case KEY_8: bibNumber = String(bibNumber + "8"); blinkLED(LED_GREEN, 100); break; case KEY_9: bibNumber = String(bibNumber + "9"); blinkLED(LED_GREEN, 100); break; case KEY_OK: bibComplete = true; blinkLED(LED_GREEN, 1000); break; case KEY_CLEAR: bibNumber = String(""); blinkLED(LED_RED, 1000); break; default: // do nothing, ignores and continues break; } //myDecoder.dumpResults(true); //For printing results. Use false for less detail myReceiver.enableIRIn(); //Restart receiver } } // I have a bib identifier Serial.print(F("Bib number read: ")); Serial.println(bibNumber); }

#activioty #activioty openiotchallenge #iot #openiotchallenge #ir #infrared #control #arduino

0 notes

espinr · 8 years ago

Text

Activ-IoTy Checkpoint: UHF RFID + RPi + Paho for Python

Update of the ActivIoTy project for the Eclipse Open IoT Challenge.

One of the most important pieces of Activ-IoTy is Checkpoints. This article is about an implementation of a checkpoint based on a IND903 UHF RFID Reader, connected to a Raspberry Pi (or a Intel Up Squared). The reader control and the checkpoint will be coded in Python. The MQTT Client is implemented using the Paho Python module. The libraries can be found in the repository.

It is important to understand that each competitor taking part in a competition is identified by two unique IDs: 1. the bib number: alphanumeric string that follows the organizers standards (it could be an alias, surname or bib number). In this case, an unsigned integer; and 2. an EPC (Electronic Product Code) corresponding to the RFID tag attached to their bib number. This tag follows the EPC C1G2 (Class 1 Generation 2) standard.

IND903 UHF RFID Reader

This checkpoint of ActivIoTy is configured to read RFID Gen 2 tags that identify competitors. In the case of running races, competitors will wear paper bib numbers with RFID tags attached to them.

The chosen solution is based on the IND903 UHF RFID reader module. The IND903 is a small reader module, operating in the 902-928MHz (US) frequency and 865.6-867.6MHz (Europe). It works over the EPC global UHF Class 1 Gen 2 (ISO 18000-6C) protocol through a USB serial Interface. The operating voltage (DC 3.5V – 5V), operating temperature (-20°C to +70°C), performance (50 tags per second during an inventory, and a label buffer of 200 labels), and low consumption (110mA at 3.5V – 18 dBm Output，25°C) makes this module appropriate for the requirements of the project.

Although the reader has a reading buffer, this pilot will use a command to perform real-time tags inventory. The default baud rate is 115200bps. Thus, at 115200 baud there is 86 microseconds between characters transmitted through the serial port. For this pilot we don't need more efficiency.

The EPC C1G2 RFID tags will store data that identifies competitors. The memory bank of tags stores several information fields, but the most interesting for us is the 96-bit EPC (Electronic Product Code). These unique codes are the ID for the athletes in the competition.

RFID Reading Serial Protocol

The RFID module implements a specific USB serial protocol. It brings along specific configuration software, and a SDK to control all the complete functions of the reader. The SDK is designed for the Windows platform and works nicely, but ActivIoTy aims at running on multi-platform devices. So, I had to build my own drivers adapted to the project needs.

Commands and responses consist of byte streams. The lengths of the streams are variable, and the packets are checked by checksum (function in the specs.

The pilot will implement some basic commands required for the project: * cmd_name_real_time_inventory: Inventory tags in real time mode. * cmd_name_get_firmware_version: Get firmware version. * cmd_name_set_work_antenna: Set working antenna.

The rest of the commands may be implemented afterwards. The reader will be configured in advance (operating frequency, address, etc.) with fixed values.

Sample of communication protocol during the operation of the reader:

Command 0x72 (get firmware version) sent to the reader:

> A0 03 01 72 EA

These 5 bytes correspond to: * A0: head (fixed) * 03: Length (starting from the third byte) * 01: Reader’s address * 72: Command * EA: Checksum (check all the bytes except itself)

The reader responds with this packet:

so, *v1.9* * `DE`: Checksum

In order to set the working antenna (in our case, it is the #01) we send the following packet:

Set antenna (antenna 1):

> A0 04 01 74 00 E7

A0: head (fixed)

04: Length

01: Reader’s address

74: Command

00: Data (00 = antenna #01)

E7: Checksum

The reader responds:

A0 04 01 89 01 D1

The reader reads a tag (one or several times). In case the reader detects a tag more than once the reader only send one packet. This is the sample packet we send to the RFID reader:

A0 0A 01 89 00 00 1D 00 00 00 01 BC

Bytes 10 and 11 represent the number of tags read. 00 01 one tag read.

Check the full specs of the IND903

Coding in Python

The driver to control the UHF RDIF module has been written in Python (v3 because of new features to work with raw bytes). This language was chosen to be able to operate with bytes easily. It is a lightweight language with multi-platform support and it's very easy to understand.

You can find all the Python classes and scripts in the repository. As mentioned before, not all commands are implemented but there are some classes that can be used to create packets, including checksums.

Do not rely too much on the scripts I created. Believe me, I tried my best, but it was my first experience with this language :)

The final running script will create a Checkpoint object that includes a Ind903Reader instance. The workflow will always be:

Set-up (set ID, address, port, etc.)

Check NTP time (set offset to the internal time clock)

Open serial USB port

Publish 'ready' message to the MQTT broker

Start inventory of tags (infinite loop: set antenna, start inventory, process reading) 5.1. Once a EPC is read, a message is published to the MQTT queue.

NTP client with ntplib module

ntplib module will is used to managing timestamps. The client will make a call during the setup to store the offset of the NTP time and the internal clock time.

ntplib is very intuitive and can be installed with PyPI pip3 install ntplib.

The implementation is pretty straightforward:

c = ntplib.NTPClient() response = c.request('europe.pool.ntp.org', version=3)

Implementation of a MQTT Client with Paho for Python

The MQTT client is implemented using the Python Paho module. Paho for Python provides a client class with support for both MQTT v3.1 and v3.1.1 on Python 2.7 (or 3.x in this case).

It can be installed easily via PyPI (pip install paho-mqtt or pip3 install paho-mqtt in my case).

This module provides some helper functions to make publishing one off messages to an MQTT server very straightforward. Publishing to the MQTT queue is as easy as this:

import paho.mqtt.publish as publish publish.single(“mytopic", “my message", hostname="test.mosquitto.org")

Since the platform will use two fixed topics (ready to indicate a checkpoint has joined the network and is in operation, and checkin to report competitor's presence or arrival at the checkpoint.

Before the device starts the inventory of competitors, the script publish a ready message, informing about the checkpoint station, following this structure:

message = { ‘checkpoint’ : ‘ID-CHECKPOINT’, ‘timestamp' : ‘NTP-UNIX-TIMESTAMP’, }

Once a competitor checks in (their identifier is detected), the RFID reader publish a checkin message to the MQTT broker with these data:

message = { ‘checkpoint’ : 'ID-CHECKPOINT’, ‘timestamp' : ‘NTP-UNIX-TIMESTAMP’, ‘competitor’ : { ‘epc’ : ‘HEX-ID-RFID-TAG' } }

Testing, testing

And, this is me, having fun and testing the system with real RFID tags in the kitchen:

#activioty #activioty openiotchallenge #iot #internet of things #RFID #python #uhf #paho

0 notes

espinr · 8 years ago

Video

youtube

High level explanation of the objectives of Activ-IoTy, a personal project being developed for the Open IoT Challenge.

I had no idea of IoT, so this is the challenge itself. Anyway, I love the concept of Open Source Hardware and I like rapid prototyping and development, so this is a perfect excuse to start.

#openiotchallenge #activioty openiotchallenge #activioty #iot #eclipse #open source #standards #timekeeping #racing #running #rfid

0 notes

espinr · 8 years ago

Text

MQTT, Cornerstone of Activ-IoTy

Update of the ActivIoTy project for the Eclipse Open IoT Challenge.

Activ-IoTy is based on the pub-sub paradigm. This means that some components (publishers or senders) send messages to a central point, called broker, that delivers the messages to other components (subscribers or receivers) interested in receiving the information send by the publishers.

Just as a reminder of what the system does:

Checkpoints registers { competitor's ID + timestamp }. So, these are the publishers

Controller(s) collect the information sent by Checkpoints and process the information. So, this is one of the subscribers.

Other subscribers may perform other activities (i.e., visualizations, integration with third party services, etc.)

MQTT

> It's a good choice for systems where we need to share small code messages, and the network bandwidth may be limited

This mechanism is implemented using MQTT 3.1.1, an OASIS standard that is widely adopted by industries and IoT solutions. MQTT is a light-weight protocol suitable for Activity, due to the flexibility requirements regarding the conditions where Checkpoints operates. MQTT guarantees connections with remote locations. It's a good choice for systems where we need to share small code messages, and the network bandwidth may be limited. So, it's a perfect choice for ActivIoTy.

MQTT ensures the data is delivered properly to subscribers. The broker is in charge of delivers messages, according to three Quality of Service (QoS) levels:

QoS 0: At most once delivery. No response is sent by the receiver and no retry is performed by the sender. This means that the message arrives at the receiver either once or not at all.

QoS 1: At least once delivery. This quality of service ensures that the message arrives at the receiver at least once.

QoS 2: Exactly once delivery. This is the highest quality of service, for use when neither loss nor duplication of messages are acceptable.

Since ActivIoTy needs to satisfy completeness and integrity of data gathered by Checkpoints, the chosen QoS level is QoS 1 (At least once delivery). QoS 2 would be also adequate but it increases the overhead of communications. There may be duplicates, that will be filtered by receivers without too much cost associated.

Topics and Messages

Application messages within the MQTT queue are related to topics. So, both sender and receivers may work with messages organized by topics. Topics follow a hierarchical structure (e.g. bedroom to indicate all messages related to a bedroom; bedroom/temperature and bedroom/humidity for specific messages sent by sensors in the bedroom). This that is useful for subscribers to filter and receive only the information they need. Forward slash (‘/’ U+002F) is used to separate each level within a topic tree structure.

Checkpoints generate two kind of application messages:

1- Readiness messages (topic ready). They are sent once the Checkpoint is initialized and connected to the network. 2- Check-in messages (topic checkin). They are sent after the check-in of a competitor.

The definition of topics may be as complex as we want, In my case, the schema of topic names is based on the id of the checkpoint and the type of message. This is: {checkpointID}/ready and {checkpointID}/checkin.

Receivers may be subscribed to topics in a flexible way, using wildcards. For instance, +/ready to subscribe to all ready topics from all senders.

Messages will be encoded as UTF-8 in JSON format as follows:

Topic +/ready:

{ "checkpoint" : { // ID required "id" : "…", // Optional metadata "name" : "…", "description" : "…", "geo" : { "lat" : 0.000, "lng" : 0.000, "elevation" : 0.000 }, }, "timestamp" : 0000000 }

Readiness messages only require checkpoint identifier and timestamp (the Unix time of the device after synchronizing with a NTP server).

Topic +/checkpoint:

{ "checkpoint" : { "id" : "…", // Required "geo" : { "…" } // (Optional) It can be included in case the checkpoint is on the move }, "bib" : "…", // Either 'bib' or 'epc' are required "epc" : "…", "timestamp" : 0000000, // Required }

Implementation with Mosquitto and Paho

Eclipse Mosquitto is an open source message broker that implements the MQTT protocol versions 3.1 and 3.1.1. This server is really stable and lightweight so it can be installed in any kind of computer, even on your Raspberry Pi.

Installation and configuration of the Mosquitto Broker is really easy (check official page for instructions). For this pilot, I installed the Mosquitto Broker on a UP² Squared I received for this challenge –thanks, it's great!

I installed it from the Mosquitto Debian Repository with apt just in a few steps:

After connecting via SSH with the UP² Squared.

wget http://repo.mosquitto.org/debian/mosquitto-repo.gpg.key sudo apt-key add mosquitto-repo.gpg.key

Repository available to apt:

cd /etc/apt/sources.list.d/ sudo wget http://repo.mosquitto.org/debian/mosquitto-wheezy.list

Update apt information and install Mosquitto

apt-get update apt-get install mosquitto

As a summary of the configuration of this broker:

Configuration file: /etc/mosquitto/mosquitto.conf

Logs: /var/log/mosquitto/mosquitto.log

Mosquito service can be managed with this command:

sudo service mosquitto [start|stop|restart]

The listener by default will open the 1883 port. Through the configuration file, we can set up the server. There is a sample file with all options listed and documented at /usr/share/doc/mosquitto/examples/mosquitto.conf.

There, we can configure all the security specifications of our broker (user/password files, SSL/TLS support, bridges, etc.).

For this pilot we don't include security measures but communications should be private and secured in the production environment.

In case we want to test the broker, we can also install the pub-sub clients anywhere.

apt-get install mosquitto-clients

Opening several terminal sessions, we can simulate subscribers (mosquitto_sub) and publishers (mosquitto_pub) with just these options: * -h (hostname) * -t (topic) * -m (message)

Do not forget to open that port on your firewall to make it available from outside your network.

Paho

Checkpoints are implemented using Eclipse Paho. Paho provides open-source client implementations of MQTT protocols for any platform. Paho implementations may be deployed on all checkpoint modules of ActivIoTy: Raspberry + Python; UP2 Squared + Python/embedded C; Arduino + embedded C).

Paho is also the base for the implementation of the Controller (subscriber), coded in Node.js.

#activioty #openiotchallenge #iot #activioty openiotchallenge #mqtt #mosquitto #paho #eclipse

0 notes