Proxy Integration

Some websites use anti-bot protection, restrict server-based requests, or rely on JavaScript rendering that can prevent a normal Direct Fetch from retrieving the expected price.

PriceFetchWP includes Proxy Integration, allowing individual URL Entries to use supported third-party providers when a normal request is not sufficient.

Proxy providers may help with blocked requests, different network routing, anti-bot handling, or browser-style page rendering.

– Why Proxy Integration Might Be Needed

By default, PriceFetchWP uses a Direct Fetch to retrieve external page data.

However, some websites may:

• Block automated or server-based requests
• Restrict certain hosting IP addresses
• Use anti-bot protection systems
• Require JavaScript before pricing becomes available
• Return different content to a server than to a normal browser

In these situations, a Proxy provider may be able to retrieve content that a Direct Fetch cannot.

Before enabling a Proxy, run Test Entry to determine what is happening with the normal request.

– Supported Proxy Providers

PriceFetchWP currently supports:

• PhantomJSCloud — browser-rendered page fetching with JavaScript support
• Scrapfly — proxy, anti-bot, and optional JavaScript-rendering services

Each provider requires its own third-party account and API credentials.

– How It Works

When Proxy Integration is enabled for a URL Entry:

1. PriceFetchWP sends the external request through the selected Proxy provider
2. The provider retrieves and processes the external page
3. The resulting page content is returned to PriceFetchWP
4. PriceFetchWP applies the configured selectors
5. Any matching price is processed normally

Proxy providers are configured per URL Entry.

This means websites that work correctly using Direct Fetch can remain direct while only the websites that need a Proxy use one.

– When to Use Proxy Integration

Consider using a Proxy when:

• Direct Fetch returns an HTTP or access error
• The external website blocks your WordPress server
• Test Entry returns a challenge or blocked page
• The price is visible in your browser but missing from the fetched content
• The website requires browser-style JavaScript rendering
• A supported Proxy successfully returns the expected page when Direct Fetch cannot

A selector failure by itself does not necessarily mean a Proxy is required.

If the page was retrieved successfully, first check the selector and use See raw HTML to confirm whether the expected price exists in the content PriceFetchWP received.

– How to Set It Up

A Proxy is a third-party service. You must create an account with the provider and obtain the required API credentials.

Step 1 — Configure the Provider

Go to:

PriceFetchWP → Settings → Proxy Integration

Select either:

• PhantomJSCloud
• Scrapfly

Enter the provider API key and use Verify to test it.

Once verification succeeds, click Quick Save to store the Proxy Integration settings.

Step 2 — Enable Proxy for a URL Entry

Find the URL Entry that requires the Proxy and open: Advanced Options

Then:

1. Enable Proxy
2. Select the desired provider
3. Save your settings

When active, the URL Entry displays a status indicator such as:

Proxy: PhantomJSCloud

or:

Proxy: Scrapfly

beside the entry number.

Affiliate Integration

Affiliate Integration and Proxy Integration cannot be active on the same URL Entry.

Affiliate mode retrieves pricing through the selected provider API rather than fetching the external webpage, so a Proxy is not used for that entry.

– Choosing the Right Provider

Which provider works best can depend on the external website, the type of protection or rendering required, provider pricing, and your own preference.

PhantomJSCloud

PhantomJSCloud provides browser-rendered page fetching and can be useful when a website requires JavaScript or behaves differently from a normal server request.

PriceFetchWP includes options such as:

• Finish at DOM Ready
• Ignore images

For configuration details, see: Using PhantomJSCloud Proxy

Scrapfly

Scrapfly provides proxy services, anti-bot handling, and optional JavaScript rendering.

PriceFetchWP supports options including:

• Anti-Scraping Protection (ASP)
• JavaScript Rendering

Scrapfly requests may consume different amounts of provider credits depending on the resources required for a particular website.

For configuration details, see: Using Scrapfly Proxy

– Proxy Trade-Offs

Pros

A Proxy provider may:

• Help retrieve websites that block normal server requests
• Provide different network routing
• Provide browser-style JavaScript rendering
• Improve compatibility with websites using stronger anti-bot protection

Cons

Proxy requests:

• Are usually slower than Direct Fetch
• May consume paid third-party credits or usage
• Depend on an external service
• Do not guarantee compatibility with every website

A Proxy request is only needed when PriceFetchWP needs to perform a new external fetch.

While a valid Price Cache result is available, PriceFetchWP can use the cached price instead of making another request through the Proxy.

Once the cache expires, the next applicable price request may use the configured Proxy provider again.

– Best Practices

• Only enable when needed

Use Direct Fetch whenever it reliably retrieves the expected price.

A Proxy adds another service to the request process and should generally be enabled only for URL Entries that require it.

• Test before enabling permanently

Use Test Entry to confirm that the Proxy actually improves the result.

Verify:

• Fetch status
• Fetch mode
• Proxy provider
• Selector matches
• Regular and Sale Price results
• Any reported errors

If the Proxy retrieves the page but no price is found, use See raw HTML to inspect the content returned to PriceFetchWP.

• Combine with caching

Use an appropriate Price Cache duration to reduce repeated external Proxy requests.

Cached prices can be reused until the configured cache expires.

For more information, see: Price Cache

• Use Delay Price for better UX

Proxy requests can take longer than Direct Fetch.

Delay Price Load allows the page to continue loading while PriceFetchWP processes the price request separately.

It can also help prevent full-page or CDN caching from locking an older displayed price into the page.

PriceFetchWP’s own Price Cache still applies to delayed requests.

⚠️ Common Issues

• Price still not loading

Run Test Entry first.

Check that:

• The Proxy API key is configured correctly
• The expected Proxy provider appears as the fetch mode
• The correct Proxy provider is enabled for the URL Entry
• The URL Entry is not paused
• The configured selector is correct
• The expected price exists in the returned content

If the Proxy request succeeds but the price is still not found, use See raw HTML to inspect what the provider returned.

You may also need to review the provider-specific settings:

• Using PhantomJSCloud Proxy
• Using Scrapfly Proxy

• Proxy request is slow

Proxy requests normally take longer than Direct Fetch because an external provider must retrieve and process the page before returning it to PriceFetchWP.

Browser rendering and additional anti-bot handling can increase that time further.

Once a valid price is stored in the Price Cache, PriceFetchWP can reuse that cached result until another external fetch is required.

• Direct Fetch works but Proxy is enabled

If Direct Fetch reliably retrieves the correct price, there is generally no reason to continue using a Proxy for that URL Entry.

Using Direct Fetch where possible keeps the request simpler and avoids unnecessary third-party usage.

Next Steps ───

For more information, continue with:

• Using PhantomJSCloud Proxy
• Using Scrapfly Proxy
• Testing a URL Entry
• Price Cache
• Blocked Requests
• Logging & Debugging