How do Price Comparison Engines Gather their Data

21 September 2017 |

How do Price Comparison Engines Gather their Data

In today’s competitive environment, eCommerce players use every opportunity to attract potential consumers. One of such opportunities is listing on price comparison websites.

Comparison shopping engines gather information about products and services, their prices, descriptions, features, etc., and then show that combined information on one page as a response to visitors’ search request. Thus, buyers can investigate easily and compare products they are looking for based on such criteria, like price, features, and shipping options, etc. so that they can find the perfect offer. As to eCommerce merchants, they get the possibility to find new customers, raise sales, and outdo their competitors.

To compare data, you need to get it first. Getting and updating data is probably the most important and challenging task for price comparison engines. So how do they collect the product data? Basically, there are three options for gathering various product information for price comparison engines.

1. Getting data from merchants’ feeds

Price comparison websites can establish an agreement with a store or a company so that they can get the data directly from retailers’ web feeds. What is more, they can also become an affiliate partner and get a referral commission.

Pros:

  • You don’t need to implement any technology to get info
 

Cons:

  • It demands a lot of manual work
  • Inability to make real-time updates
  • Getting the needed information from thousands of merchants separately will get you nowhere
  • Managing information coming in multiple formats from multiple sources is a hell
 

2. Crawling

Using crawling bots, which regularly gather information from websites for further processing. Thus, it is possible to get accurate information about products, which is available on the site. However, many websites have special anti-crawling mechanisms, which block users, that make too many requests.

Pros:

  • You can get rich and reliable data of a product;
  • You don't depend on merchants to provide you with the data.
 

Cons:

  • Expensiveness;
  • Anti-crawling mechanism on websites, restrict from gathering info;
  • Difficulties in writing a crawler for each of the websites;
  • Inability to always get an up-to-date data, as crawling takes quite a long time.
 

3. API integration

You can integrate with shopping carts and get all the data needed for accurate product comparison via API requests: Prices, variants, colors, images, descriptions, etc. This method is very convenient and easy to work with, but the API integration is not so simple. Taking into account the specificities of every shopping platform, developing multiple integrations is a difficult and expensive process. However, it can be much easier if integrating via API2Cart. It is a service that allows accessing all data assets on 30+ shopping platforms. It offers a unified API to connect to different shopping carts simultaneously including such leaders as Magento, WooCommerce, Bigcommerce, OpenCart, Shopify and many others.

Pros:

  • It saves a lot of time on development;
  • Allows to integrate comparison price engines with multiple shopping carts easily;
  • Allows to gather products’ images, descriptions, variants, and options from stores, and process it within your system.
 

Cons:

  • It is not free. But, as Benjamin Franklin said, “Time is money.” So invest wisely.

If you run a price comparison engine and have questions about API2Cart functionality and how it can help your business, don’t hesitate to contact us. Schedule a call with our representative or register an account and try how API2Cart will work for your business.