#jquery add row to table after specific row | Explore Tumblr posts and blogs

atlassianreleasenotes · 6 years ago

Text

Jira Service Desk 4.0.0 Release Notes

We're excited to present Jira Service Desk 4.0.

In our latest platform release, Jira Service Desk harnesses an upgraded engine that provides faster performance, increased productivity, and greater scalability. Dive in and discover all the goodies awaiting your service desk teams.

Highlights

Lucene upgrade

jQuery upgrade

Everyday tasks, take less time

Accessible priority icons

New options in advanced search

AUI upgrade

AdoptOpenJDK support

End of support for fugue

New customer portal in the works

Resolved issues

Get the latest version

Read the upgrade notes for important details about this release and see the full list of issuesresolved.

Compatible applications

If you're looking for compatible Jira applications, look no further:

Jira Software 8.0 release notes

Lucene upgrade to power performance

A welcome change we inherit from Jira 8.0, is the upgrade of Lucene. This change brings improvements to indexing, which makes it easier to administer and maintain your service desk. Here’s an overview of what to expect:

Faster reindexing

Reindexing is up to 11% faster. This means less time spent on reindexing after major configuration changes, and quicker upgrades.

Smaller indexes

Index size has shrunk by 47% (in our tests, the index size dropped from 4.5GB to 2.4GB). This means a faster, more stable, Jira that is easier to maintain and troubleshoot.

jQuery upgrade

Outdated libraries can lead to security vulnerabilities, so we've upgraded jQuery from version 1.7.2 to 2.2.4. This new version includes two security patches, and the jQuery migrate plugin 1.4.1 for a simplified upgrade experience. Learn more

Everyday tasks, take less time

We know how important it is for agents to move through their service desk with ease, and for admins to scale their service desk as their team grows. The back end upgrade combined with a bunch of front end improvements, makes work more efficient for admins, agents, and customers alike:

Viewing queues is twice as fast in 4.0 compared to 3.16.

Opening the customer page is 36% faster in 4.0 compared to 3.16

Project creation for an instance with 100 projects, is twice as fast in 4.0 compared to JSD 3.15.

Project creation for an instance with up to 200 projects, is six times faster in 4.0 compared to JSD 3.15.

We'll continue to take performance seriously, so that your service desk team can get the most out of their day!

Accessible priority icons

We’ve updated our priority icons to make them more distinctive and accessible. These changes allow Jira users with color blindness, to instantly recognize the relationship between icons, and their level of importance. Here’s a comparison of old and new icons that you'll be seeing around Jira Service Desk:

New options in advanced search

Check out these new options for when you are using advanced search:

Find contributors and date ranges (updatedBy)

Search for issues that were updated by a specific user, and within a specified date range. Whether you're looking for issues updated in the last 8 hours, two months, or between June and September 2017 – we've got you covered. Learn more

Find link types (issueLinkType)

Search for issues that are linked with other issues by particular link types, like blocks or is duplicated by. This will help you quickly find any related blockers, duplicates, and other issues that affect your work. Learn more

Atlassian User Interface (AUI) upgrade

AUI is a frontend library that contains all you need for building beautiful and functional Atlassian Server products and apps. We've upgraded AUI to the more modular 8.0, making it easier to use only the pieces that you need.

AdoptOpenJDK comes to Jira

Oracle stopped providing public updates for Oracle JDK 8 in January 2019. This means that only Oracle customers with a paid subscription or support contract will be eligible for updates.

In order to provide you with another option, we now support running Jira Service Desk 4.0 with AdoptOpenJDK 8. We'll continue to bundle Jira Service Desk with Oracle JDK / JRE 8.

End of support for fugue

In Jira Service Desk 4.0, we've removed com.atlassian.fugue, and updated our APIs to use Core Java Data types and Exceptions.

You can read the full deprecation notice for next steps, and if you have any questions – post them in the Atlassian Developer Community.

Get ready for the new customer portal experience

We're giving you more ways to customize your help center and portals, alongside a fresh new look that will brighten up your customer's day. This first-class, polished experience will be available in the not too distant future, so keep an eye on upcoming release notes. In the meantime, here’s a taster…

Added extras

4-byte characters

Jira now supports 4-byte characters with MySQL 5.7 and later. This means you can finally use all the emojis you've dreamed about!

Here's a guide that will help you connect Jira to a MySQL 5.7 database.

Add-ons are now apps

We're renaming add-ons to apps. This changed in our Universal Plugin Manager some time ago, and now Jira Service Desk follows suit. This change shouldn't really affect you, but we're letting you know so you're not surprised when seeing this new name in Jira administration, and other pages.

Issues in Release

Fixed: The SLA "within" field keeps increasing when the SLA is paused

Fixed: The SLA information is not included in the Excel and HTML export

Fixed: Anonymous comments cause display issues in Activity section in Issue Navigator

Fixed: Portal Settings - Highlight and Text on highlights values are swapped on subsequent edits

Fixed: Inconsistent JIRA Service Desk REST API Pagination

Fixed: JSON export doesn't differentiate public from internal comments

Fixed: Asynchronous cache replication queue - leaking file descriptor when queue file corrupted

Fixed: New line character in issue summary can break issue search detailed view

Fixed: Jira incorrectly sorts options from Select List custom fields with multiple contexts

Fixed: DefaultShareManager.isSharedWith performs user search 2x times

Fixed: Unknown RPC service: update_security_token

Fixed: Copying of SearchRequest performs user search

Fixed: Data Center - Reindex snapshot file fails to create if greater than 8GB

Fixed: The VerifyPopServerConnection resource was vulnerable to SSRF - CVE-2018-13404

Fixed: XSS in the two-dimensional filter statistics gadget on a Jira dashboard - CVE-2018-13403

Fixed: DC index copy does not clean up old files when Snappy archiver is used

Fixed: SQL for checking usages of version in custom fields is slow even if no version picker custom fields exist

Suggestion: Improve FieldCache memory utilisation for Jira instances with large Lucene

Fixed: Search via REST API might fail due to ClassCastException

Fixed: Upgrade Tomcat to the version 8.5.32

Suggestion: Webhook for Project Archive on JIRA Software

Fixed: Deprecate support for authenticating using os_username, os_password as url query parameters

Fixed: JQL input missing from saved filter

Fixed: CachingFieldScreenStore unnecessary flushes fieldScreenCache for create/remove operation

Suggestion: Favourite filters missing in Mobile browser view

Fixed: DC node reindex is run in 5s intervals instead of load-dependent intervals

Suggestion: Don't log Jira events to STDOUT - catalina.out

Fixed: Adding .trigger-dialog class to dropdown item doesn't open a dialog

Fixed: Issue view/create page freeze

Suggestion: Add Additional Logging Related to Index Snapshot Backup

Suggestion: Add additional logging related to index copy between nodes

Fixed: JIRA performance is impacted by slow queries pulling data from the customfieldvalue table

Fixed: Projects may be reverted to the default issue type scheme during the change due to race condition

Suggestion: As an JIRA Datacenter Administrator I want use default ehcache RMI port

Fixed: Two dimensional filter statistics gadget does not show empty values

Suggestion: Embed latest java critical security update (1.8.0.171 or higher) into the next JIRA (sub)version

Fixed: JIRA inefficiently populates fieldLayoutCache due to slow loading of FieldLayoutItems

Suggestion: Provide New look Of Atlassian Product For Server Hosting As Well

Fixed: Startup Parameter "upgrade.reindex.allowed" Not Taking Effect

Fixed: Remote Linking of Issues doesn't reindex issues

Fixed: Index stops functioning because org.apache.lucene.store.AlreadyClosedException is not handled correctly

Suggestion: Add Multi-Column Index to JIRA Tables

Fixed: 'entity_property' table is slow with high number of rows and under high load

Fixed: JIRA Data Center will skip replication operations in case of index exception

Fixed: Improve database indexes for changeitem and changegroup tables.

Suggestion: CSV export should also include SLA field values

Fixed: Priority icons on new projects are not accessible for red-green colour blind users

Fixed: Two Dimensional Filter Statistics Gadget fails when YAxis is a Custom Field restricted to certain issue types

Fixed: Unable to remove user from Project Roles in project administration when the username starts with 0

Suggestion: The jQuery version used in JIRA needs to be updated

Fixed: MSSQL Selection in config.sh Broken (CLI)

Suggestion: Upgrade Lucene to version 7.3

Fixed: User Directory filter throws a 'value too long' error when Filter exceeds 255 chars

Suggestion: Add support for 4 byte characters in MySQL connection

Fixed: xml view of custom field of multiversion picker type display version ID instead of version text

Fixed: Filter gadgets take several minutes to load after a field configuration context change

Suggestion: Ship JIRA with defaults that enable log rotation

Fixed: The xml for version picker customfield provides the version id instead of the version name

Suggestion: JQL function for showing all issues linked to any issue by a given issue link type

Suggestion: Ability to search for issues with blockers linked to them

Suggestion: Reduce JIRA email chatiness

Source

#jira #jiraservicedesk #jiraserver #atlassian #atlassianreleasenotes

0 notes

sdesignermagazine · 8 years ago

Photo

Tutorial: Creative Spitscreen Layout with CSS3 Animations (Pt. 2)

A few weeks ago, I demonstrated how to build a split-screen website layout using CSS flexbox and viewport units that offers an alternative way to present a brand’s featured content. Clicking on one side or the other navigates further into the site without a page load, using CSS transitions and 3d transforms. This week, I’ll show you how to use these same techniques to add animation and movement to the content and buttons.

Check out Part 1

See the Pen CSS Responsive Full Screen Duo Layout With Animated Overlay by Markus Sladek (@SDesignerMagazine) on CodePen.

Structure If you followed along for part one, you should already have created the overlay, or wrapper for your content and added a jQuery snippet to animate its entrance using Animate.css classes. Here is a refresher on which pieces we start with, using the left or west side as an example:

<div id="west-overlay" class="overlay"> content goes here </div>

$("#west .content").click( function() { $("#west-overlay").addClass("animated fadeInLeft open").one('webkitAnimationEnd mozAnimationEnd MSAnimationEnd oanimationend animationend', function(){ $(this).removeClass("animated fadeInLeft"); }); });

In keeping with the two-column style of the design, this page will also use flexbox to create a single column within which the content can be layed out:

<div id="west-overlay" class="overlay"> <div class="grid"> <div class="column"> <img src="photo-url"/> </div> <div class="column"> <div class="product-content"> <h1>Rainbow Chasers</h1> <p>...</p> <div class="price"> <span class="currency">$</span> <span class="amount">89.99</span> </div> <a href="#" class="buy-button"><span>Buy Me</span></a> <div class="size-guide-link underline-link">Size guide</div> </div> </div> </div> <div class="close-icon"> <div></div> <div></div> </div></div> </div>

Within the overlay, we have created a grid wrapper to hold our flex items, starting with two columns. The left column holds an image, and the right column holds our page content, which in this case is a heading, description, price and call to action for a featured product. Spans are used to allow us to style specific text elements separately without affecting how the text naturally sits in its parent element.

Finally, a close-icon is created in pure CSS and placed within the main overlay, consisting of one main div and two inner divs to create the X.

Styling

The existing grid rule used for our main layout is applied to setup our flexbox.

.grid { position: relative; width: 100vw; height: 100vh; display: flex; flex-direction: row; flex-wrap: wrap; }

To vertically center the content responsively, we will use table. This could also be done with flexbox inception, but table is much simpler. Learn more about vertical centering here.

.overlay .column{ display: table; height: 100%; } .product-content{ font-size: 1.4rem; padding: 10%; display: table-cell; vertical-align: middle; }

Animating the Content

This demo makes use of two kinds of animation – the static kind gained through transitions, and the more complex kind using keyframes. In the case of transitions, you select the property you want to transition, which can be anything from a background gradient to a transform. For animations, you select the keyframe block. The magic happens once you declare the duration length for the animation or transition.

The transition timing function is what controls the values of intermediate states between the start and end declarations. It’s optional, and defaults to ease. When dealing with simple animations and transitions, you’ve probably used ease-in-out to make link hovers, buttons and other simple movements appear more graceful, but transition is not enough to move an element along a path on a specified timeline, allowing it to change in multiple ways. That is where CSS animation is useful.

When the diamond area of our layout, selected with #west .content is clicked, our content div with the ID of #west-overlay will get three new classes: animated, fadeInLeft and open. When the animation ends, the animation classes will be removed.

The open class makes the div visible and adds a transition effect.

The animated class, provided by Animate.css, sets up the animation duration and primary rules for the movement.

The fadeInLeft class, provided by Animate.css, fades in the div and moves it into position from the left.

You can change the animation type by replacing fadeInLeft with any of the available animations in Animate.css, or with your own animation name. Here is a look at what fadeInLeft looks like using CSS animation keyframes:

@keyframes fadeInLeft { from { opacity: 0; transform: translate3d(-100%, 0, 0); } to { opacity: 1; transform: none; } } .fadeInLeft { animation-name: fadeInLeft; }

Pretty simple, right? Over a default animation-duration of one second, the overlay div will fade in and be moved into position.

The content of the overlay can be animated in the same way, using an endless combination of styles. Create another jQuery statement inside the click function for the #west .content selector:

$("#west-overlay .product-content").addClass("animated flipInY ").one('webkitAnimationEnd mozAnimationEnd MSAnimationEnd oanimationend animationend', function(){ $(this).removeClass("animated flipInY"); });

Delays and Duration

So when the diamond area is clicked, both the overlay animation and the content animation are kicked off. If you wanted your content to begin animating only after the overlay does, you would tuck it into the one function instead, just after $(this).removeClass("animated fadeInLeft");

Using what you’ve learned about using these functions, you can select any part of your content to animate it separately, such as fading in the image, sliding down the heading and then flipping in the button, for example. To add a delay for the animations, or make the animation slower or faster, the following properties can be added to your CSS rules for the element you are adding the animation class to:

.product-content { animation-duration: 3s; animation-delay: 2s; }

As you can see, the visual effects are achieved through transitions and transforms, whereas movement along a path is achieved through the additional use of CSS animation. In the next section, we’ll look at the static effects used to add more interest and life to the interactions on the page, meant to entertain our visitors and keep them more engaged.

Interactions

To get the layered look of the button, we use styling of pseudo-elements ::before and ::after, and a span for the button text so we can position it properly when the button is pressed.

A transition is added to the span to give the transform we will add on hover a smooth animation.

.buy-button{ height: 50px; width: 100%; margin: 2rem 0; position: relative; background: #eee; border: 3px solid #f9efb2; } .buy-button > span { display: block; position: relative; font-weight: 700; font-size: 1rem; text-transform: uppercase; color: #1f1f22; transition: opacity 0.35s, transform 0.35s; z-index: 10; line-height: 60px; }

The pseudo-elements are given a border and transition to match the above. They are offset using absolute positioning and sent behind our span using z-index to give the button some dimension.

.buy-button:after, .buy-button:before { content: ''; position: absolute; top: 0; left: 0; border: 3px solid #f9efb2; transition: opacity 0.35s, transform 0.35s, ease-in-out .3s; } .buy-button:before { background: #FFF; margin: 0px 0 0 0px; top: 5px; left: 5px; width: calc(100% + 2px); height: calc(100% + 2px); z-index: 1; }

The hover effect is created using translate3d to move the top face of the button a few pixels backwards. Simple!

.buy-button:hover span, .buy-button:hover:before { transform: translate3d(-4px, -4px, 0px); }

Link Underline

Next we have a subtle "animated" underline on our Size Guide link. The line is created using a pseudo-element:

.underline-link::before { content: ''; position: absolute; bottom: 0; left: 0; width: 100%; height: 1px; background: #202023; transform-origin: left; transform: scaleX(0); transition: transform .2s ease-in-out; }

The transform-origin: left declaration draws the line from the left. Remove this if you want it to grow from the center. The transform: scaleX(0) declaration establishes the starting point as 0 which we will grow to 1 on hover, and a basic ease-in-out transition is added to the transform to create a smooth "animated" look when we change its value on hover:

.underline-link.active::before, .underline-link.hovered::before, .underline-link:hover::before { transform: scaleX(1); }

Close Button

Going back to transition timing functions like ease and linear, you may have noticed that they are somewhat limited. If you want more control over the speed of the effect as it is played out, the cubic-bezier property does the trick. To put it simply, cubic-bezier allows your transition to play out along a bezier curve rather than a straight line, allowing for 4 points of control rather than two. This can be used to create more dimensional effects, or in the case of our Close Button, to give each "leg" of our X the appearance of separate animations while only applying the effect to two divs.

.close-icon div { display: block; height: 3px; position: relative; transition: transform 0.6s cubic-bezier(0.19, 1, 0.22, 1) 0s, opacity 0.4s cubic-bezier(0.19, 1, 0.22, 1) 0s; width: 36px; }

The button itself is created from our two inner divs and the ::before and ::after pseudo-elements, formed into an X using translate3d transforms.

.close-icon div::after, .close-icon div::before { content: ""; display: block; height: 100%; pointer-events: none; position: absolute; width: 100%; } .close-icon div::before { background: none repeat scroll 0 0 #34d5cf; opacity: 1; transform: translate3d(0px, 0px, 0px); } .close-icon div::after { background: none repeat scroll 0 0 #ead918; opacity: 0; transform: translate3d(100%, 0px, 0px); } .close-icon div:first-child { transform: translate3d(0px, 10px, 0px) rotate3d(0, 0, 1, -45deg); } .close-icon div:first-child::after, .close-icon div:first-child::before { transition: transform 0.6s cubic-bezier(0.19, 1, 0.22, 1) 0s, opacity 0.2s cubic-bezier(0.19, 1, 0.22, 1) 0s; } .close-icon div:last-child { transform: translate3d(0px, 8px, 0px) rotate3d(0, 0, 1, 45deg); } .close-icon div:last-child::after, .close-icon div:last-child::before { transition: transform 0.6s cubic-bezier(0.19, 1, 0.22, 1) 0.2s, opacity 0.2s cubic-bezier(0.19, 1, 0.22, 1) 0.18s; } .close-icon:hover div::before { opacity: 0; transform: translate3d(-100%, 0px, 0px); } .close-icon:hover div::after { opacity: 1; transform: translate3d(0px, 0px, 0px); }

A cubic-bezier transition is then added to the opacity and transform to get the effect of each "leg" flying in or out when changed on hover:

.close-icon:hover div::before { opacity: 0; transform: translate3d(-100%, 0px, 0px); } .close-icon:hover div::after { opacity: 1; transform: translate3d(0px, 0px, 0px); }

Finally, we need to add one last jQuery function to animate and hide the overlay when it is closed by clicking on our close button.

$("#west-overlay .close-icon").click( function() { $("#west-overlay").addClass("animated flipOutY ").one('webkitAnimationEnd mozAnimationEnd MSAnimationEnd oanimationend animationend', function(){ $(this).removeClass("animated flipOutY open"); }); });

Here we add the Animate.css class animated and our chosen animation class, detect when the animation ends with one, then remove the animation classes as well as the open class to hide the overlay and bring us back to our main layout.

Resources:

CSS Transitions

CSS Animation

Animate.css

scale()

translate3d()

transform-origin

cubic-bezier() helper

Full Screen Demo

Download File

#webdesign

1 note · View note

imapplied · 7 years ago

Text

Prototyping automated product feeds for retailers

I was recently faced with the problem of creating a series of automated product feeds for a retail client.

The requirement – deliver a feed in .csv format that contains updated stock and pricing information from 3rd party sites. In this case, from a group of supplier websites for items that aren’t held in the retailer’s stock. Thus, leaving critical information about availability missing from inventory management.

This initially cool problem quickly became a daunting one as there was almost no budget available. I would add that none of the suppliers involved would supply (or would develop) a feed; which had they been willing, would have saved me an awful lot of work.

Scraping product pages is difficult, and it’s different every time

Scraping websites with variable products (drop-down selectors, checkboxes, radio buttons) that use a random variety of XHR POST and GET requests to update information such as availability and pricing in Javascript is difficult.

Every website you tackle is almost a completely new challenge as they’re all so different. Websites with variable options doubly so.

So this article shares my methods and an approach for a number of different circumstances that I’ve encountered along the way. Some of this is of course very niche, like using SEO Tools for Excel to make POST requests to an API endpoint.

It’s niche, but technical and therefore very interesting.

The main problem is this:

How do you build a product feed on a website that has variable options?

It’s actually quite difficult because most retailers have products with variations and variation-specific data isn’t forthcoming until you (as a user) have physically selected the correct options.

Once options on the product page are selected, what happens next depends on the platform.

Requests tend to be made via XHR / Ajax request, either using a Javascript library unique to the platform or something more common like jQuery. Parameters describing the product variations are added to a request URL ready for a GET request, but sometimes those parameters are sent in the form data via a POST.

The results are received as a text, JSON or CSV response (usually JSON but often some random format) and the page is updated appropriately. Sometimes all the data is already available in JSON embedded somewhere in the page source, and the JavaScript is simply updating the DOM based on what’s already there.

So there’s an awful lot of nuance in deciphering how a retail CMS updates product pages, and if you want to grab a retailer’s data, it’s an equally nuanced procedure.

Before you do anything then, you need to start by gathering some information.

Information gathering

I always start by collecting notes.

What technology platform is this? Are there any useful clues in the robots.txt file? Is there a decent sitemap.xml file?

Where is the API documentation? A surprising number of sites just leave their API exposed by complete accident.

I’ll poke around in Chrome Dev Tools, heading straight to “Network > XHR”, and select an option on the page to see what happens. This action usually exposes how data is supplied to update the page. I also check for the HTTP method used.

GET is the typical method although you do come across POST which is a different ballgame to work with. If POST is the method, this approach will expose the endpoint to which POST requests are made and responses received.

I’ll make notes about the drop-down selectors too; the value and ID attributes in the form input element are usually the parameters needed to pre-select the options via a URL GET request.

Often there’s evidence that a product URL will accept parameters to pre-select product variations. A Google Shopping result might give up some information, or a technical question on a forum. As there are only so many CMS platforms, the chances are, after a while you’ll have worked with them all.

Take a look at this example, a set of size selectors using (styled) radio buttons:

In this particular case, everything I’m going to need is contained in the label element in the for="" attribute. Collecting this data may well be important, so it’s useful to know the XPath to these snippets of data.

In our example about, the XPath: //label/@for extracts the parameters group[5][2083].

Usefully, these parameters can be added to the end of the URL to pre-select the relevant options. For example:

Request URL (GET) /product-page.html?group%5B5%5D=2082

So I hope you can see where a process might be forming! Generally, the goal of the information gathering phase is to answer questions like:

1) Can a page be loaded with options pre-selected or is there a URL that supplies the information I’m looking for?

2) Can those option values be extracted from the product page on a first pass scrape?

3) Can I concatenate these as parameterised request URLs ready for scraping?

4) What’s the best way to schedule a scraper?

Keeping notes along the way pays dividends down the road.

URL concatenation for parameterised GET requests

Once you’ve done your research and you know what you’re aiming for, it’s time to start gathering the actual data. I like to be able to build text lists of URLs that feature the data I need so that I can scrape them one by one.

To be able to build such a list, the process looks like this:

Fetch all form values for each product URL

Concatenate URLs with each combination of values added as parameter

Save as a .txt file

Fetch all form values for each product URL

The example below is a website that displays delivery time (10 work days!) and the SKU number when one of these boxes are clicked. They’re actually styled radio buttons, but that makes no difference.

What’s important, is what happens when a radio button is selected:

This is where my XPath example from earlier comes into play. We want to collect all of the parameters, of which there may be as many as 10 per page.

I end up with something along these lines:

I’m a big fan of the custom extraction tool in Screaming Frog.

It features a proper XPath parser, which includes being able to chain XPath rules with OR operators. For my use case, this feature makes it easy to chain rules together to catch different types of pages with different selectors (dropdowns, for example).

So, I collect the values for the form inputs and the names (for example, size, colour and so on).

PS: for a general introduction on how I use Screaming Frog for content research using simple XPath examples, read this article, and more recently, Dave’s article on scraping “People Also Ask” boxes.

Concatenate URLs with each combination of values added as parameter

Whenever Screaming Frog matches more than 1 result with an XPath query, it creates multiple columns in an export like this:

Concatenating each of these parameters into actual URLs is relatively straightforward:

Although you end up with a table of URLs that can have upwards of 18 columns! This requires some thought to transpose to a list:

Here’s that formula, which requires the source to be a range (Excel Tables don’t work as a source for this part of the process).

=INDEX(transpose_master,1+INT((ROW(‘Request URL Builder’!A91:R91)-1)/COLUMNS(transpose_master)),MOD(ROW(‘Request URL Builder’!A91:R91)-1+COLUMNS(transpose_master),COLUMNS(transpose_master))+1)

=INDEX(transpose_master,1+INT((ROW(‘Request URL Builder’!A91:R91)–1)/COLUMNS(transpose_master)),MOD(ROW(‘Request URL Builder’!A91:R91)–1+COLUMNS(transpose_master),COLUMNS(transpose_master))+1)

Where ‘Request URL Builder’ is the range and ‘transpose_master’ is the target table.

The result of all of this is a source list of URLs to be saved as a .txt file for a crawler to work though to collect our data.

Making POST requests

During this project, I came across a project built with OpenCart – a lesser known open source retail CMS. It doesn’t natively support variable products of any sort, so you have to install a premium plugin called OpenStock. This enables variable options to be selected on the product page.

There is almost no useful documentation for this plugin. But, after following my investigative process, it became clear the plugin wouldn’t accept URL / parameterised GET requests.

Instead, all the parameters are sent in form data via POST – see below:

(Follow along on this site’s product page for an Industry Nine Rear Hub)

Navigating to Network > XHR > Headers reveals the POST request. Scrolling down the header information reveals the form data:

And the response, in JSON format looks like this:

When you’re trying to build a feed, discovering how this data is fetched from the server is most of the battle won.

The next challenge is to mimic the request yourself to get to the same result. For that, I use API Tester. It features a very clean and simple UI and therefore getting started is easy.

In the case of the OpenCart site, we’re actually talking about a total of 100 products, perhaps 20 of them are variation products like the one above. Given there’s no budget and consequently, no developer, I had to get creative about how to solve this problem.

Enter my old friend, SEO Tools for Excel.

SEO Tools for Excel

Now, a lot of you remember how much I used to love writing Excel posts that inevitably referred to SEO Tools for Excel from my friend, Niels Bosma. It’s €99 well spent in my opinion as it helps you solve all sorts of problems very quickly. It’s useful for prototyping things that might go on to be formally developed or just as a platform to build one-off solutions.

It’s also significantly more powerful than it used to be as the latest versions run multi-threaded requests so you can switch between tabs and keep working while it’s running a bunch of HTTP requests. Honestly if you don’t have this tool, trial it and get a licence.

Making POST requests with SEO tools for Excel

As I’ve often built prototype scrapers in Excel for a quick solution or a one-off bit of research, I’m pretty used to the =XPathOnURL() and other HTTP functions included.

This is the Global HTTP Settings dialogue inside the tool:

Clearly, there’s functionality available for POST requests in the master HTTP Settings dialogue. Similarly, the =HttpSettings() function allows for POST to be set as the request method.

Unfortunately, the HTTP Settings function doesn’t have support for a POST body. I’m hopeful that might change one day as it’ll open up a whole new world of coolness. However, when I reached out to the team, I got a really helpful response from Victor Sandberg who put together a connector to make this possible.

Connectors are relatively simple XML files that configure how SEO Tools for Excel connects to, authenticates with and parses the responses from API endpoints. There’s tremendous power in them, and for the most part, you can learn how they work from viewing the XML. Here’s the full collection from Niel’s Github.

Here’s the connector to make POST requests to an endpoint with two attributes submitted in the post body. The parser settings will extract SKU, Price and a number of other objects. You should be able to see how this works from the XML. The formula in SEO Tools for Excel looks like this:

=Dump(Connector(“Baxter.Baxter”,C2,B2,”product_option_variant_id,sku,stock,active,subtract,price,image,weight,pop,nostock”,TRUE))

=Dump(Connector(“Baxter.Baxter”,C2,B2,“product_option_variant_id,sku,stock,active,subtract,price,image,weight,pop,nostock”,TRUE))

And here’s how that looks in a table:

Fortunately, this is quite an edge case, although I’m grateful for the help from the SEO Tools for Excel team, and delighted that I’ve unlocked this skill. If you want to learn more about making connectors for SEO Tools for Excel, try this helpful Wiki page.

Briefing a developer to formalise this procedure would be an absolute walk in the park.

Now, to finish with the automation challenge.

Scheduled scrapers from URL lists

I strongly recommend Scrapinghub, a recommendation my friend David Sottimano made a few months back.

It has a visual scraping editor called Portia, while Scrapy Cloud is more intended for people who can write code. Either way, it’s inexpensive, powerful and well worth learning.

My setup runs from a .txt file list of URLs to scrape, approximately 20,000. It runs weekly and uploads a .csv file via SFTP to our server. From there, stock levels are updated to schedule as intended! Writing about this setup is very much a blog post to itself, so watch this space or have an explore for yourself.

Summary

What started as an initially quick job in Screaming Frog turned into weeks of data work.

Sometimes you find yourself in a position where you have to solve a problem that isn’t well documented or hasn’t really been tackled in this way before.

You might be asking yourself why not code your way out of the problem? That’s not the point here – before you build anything in code you really need to understand the dynamics of the problem you’re trying to solve. That’s the point of prototyping, and I’m very comfortable with using the best tools around me to get to a solution.

Scraping can get costly, fast. You can quickly get into a position where weeks of development work and server time is escalating costs beyond the economic scope of the project.

I really relish problems like this as personally, I think understanding how websites work is the most important skill we can have. Have fun testing this very niche technical process!

First Found Here

from https://www.imapplied.co.za/seo/prototyping-automated-product-feeds-for-retailers/

0 notes

atlassianreleasenotes · 6 years ago

Text

Jira Core 8.0.0 Release Notes

We're pleased to present Jira Core 8.0.

Highlights

Better email notifications

New priority icons

New options in advanced search

Faster indexing

REST API for issue type schemes

Smaller improvements to make your day

Bonus resources

Resolved issues

Get the latest version

Interested in trying a shiny new version? Don't forget to renew your active software maintenance! Renew now.

Read the upgrade notes for important info about this release and see the full list of issues resolved.

Compatible applications

If you're looking for compatible Jira applications, look no further:

Jira Software 8.0 release notes

Jira Service Desk 4.0 release notes

Better email notifications

With every issue update sent in a separate email, email notifications can get overwhelming. Digging through a pile of emails first thing in the morning is never a good idea, that’s why we’ve embarked on a mission to make notifications smarter, and less chatty.

In Jira 8.0, you can batch events that occur close together, and receive them in a single summary email. Every email like that groups events that occurred in a single issue within the 10-minute window, and is very helpful when you're doing any bulk updates, or when your teammates create issues and fill them in in one go. Here’s how a summary email looks:

Your Jira admin can enable this feature by going to (cogwheel) > System > Batching email notifications. Learn more

New priority icons

We’ve got a lot of feedback about our priority icons not being distinctive or accessible, and we have to agree with that. Finally, after countless hours of designing, user testing, and even looking for inspiration in Egyptian hieroglyphs, we’ve decided on a set of simple and clear icons that will make it easier to distinguish what needs to be done first. Here’s a comparison of old and new icons that you'll be seeing around Jira:

New options in advanced search

Here are some new options that you can use when searching for issues with advanced search:

Find authors (updatedBy)

Search for issues that were updated by a specific user, within a specified time range. Whether you're looking for issues updated in the last 8 hours, two months, or between June and September 2017 - we've got you covered. Learn more

Find link types (issueLinkType)

Faster indexing

One of the big changes coming in Jira 8.0 is the upgrade of Lucene, Jira’s search-based subsystem engine. Though not visible at first glance like other user features, this change brings significant improvements to indexing, which makes it easier to administer and maintain your Jira instance. Here’s an overview of how this change affects indexing:

All of the following performance improvements are based on the Jira 8.0 vs Jira 7.6 comparison.

Reindexing Jira takes much less time Reindex duration is 71% faster (in our tests, a 3h reindex dropped to 53 minutes.) This means less time spent on reindexing after major configuration changes, and quicker upgrades.

Jira indexes are smaller, and easier to maintain Index size has shrunk by 48% (in our tests, the index size dropped from 19GB to 9.9GB.) This means a faster, more stable Jira that is easier to maintain and troubleshoot.

Jira performance is stable over time No need to reindex often, as performance stays consistent and doesn’t degrade with time.

Jira is more stable and predictable Memory issues that occurred with intensive search functions, like very complex JQLs, are nonexistent in our 8.0 tests.

For more detailed performance results, see Performance and scale testing.

REST API for issue type schemes

Jira 8.0 brings REST APIs to let you manage issue type schemes outside of the user interface. As a Jira admin, you can perform the following actions:

Create an issue type scheme and associate it with projects

Retrieve individual or all issue type schemes, together with the associated projects

Change or remove projects associated with a scheme

Delete issue type schemes

For more info, see Issue type schemes in Jira REST API docs.

Small improvements to make your day

4-byte characters

Jira now supports 4-byte characters with MySQL 5.7 and later. This means you can finally use all these emojis you've dreamed about! Here's a guide that will help you connect Jira to a MySQL 5.7 database. You can find more details about this change in this suggestion.

Add-ons are now apps

We're renaming add-ons to apps. This has already changed in our Universal Plugin Manager some time ago, and now Jira follows suit. This change shouldn't really affect you, but we're letting you know so you're not surprised when seeing this new naming in the Jira administration and other pages.

Bonus resources

Performance and scale testing

With every Jira release, we’re publishing a performance and scaling report that compares performance of the current Jira version with the previous one. The report also contains results of how various data dimensions (number of custom fields, issues, projects, and so on) affect Jira, so you can check which of these data dimensions should be limited to give best results when scaling Jira. You can get the report for Jira 8.0 here.

Security overview and advisories

We’ve created a document for system administrators who want to evaluate the security of the Jira application. The document answers a number of questions that commonly come up when customers ask us about the security of our product. It will also list any security advisories published for Jira 8.0. You can view it here.