Executing JavaScript with Playwright in Python

Playwright is a powerful browser automation library that lets developers test and automate websites using multiple programming languages, including Python and JavaScript. In this article, we will explore how to execute JavaScript code within a webpage using the Playwright library in Python. This is useful for testing specific client-side functionalities or scraping data from websites that rely heavily on JavaScript.

To get started, you need to have Python installed on your machine and also need to set up Playwright for Python. Make sure to have pip installed to facilitate the installation of necessary packages.

Installation

First, install Playwright by running the following command in your terminal or command prompt:

pip install playwright

After installing Playwright, you need to download the browser binaries that Playwright uses to run the tests. Execute the following command:

playwright install

Basic Setup

Once installation is complete, you can start writing scripts. Let's start with a simple script to launch a browser, navigate to a webpage, and close the browser:

from playwright.sync_api import sync_playwright

def run(playwright):
    browser = playwright.chromium.launch()
    page = browser.new_page()
    page.goto('https://example.com')
    browser.close()

with sync_playwright() as playwright:
    run(playwright)

This script will open the Chromium browser, navigate to 'https://example.com', and then close the browser. Asynchronous APIs are also available if you prefer working with asyncio.

Executing JavaScript

Now, let's delve into executing custom JavaScript within the webpage context using Playwright. You can evaluate JavaScript by using the page.evaluate() method. Here's an example:

from playwright.sync_api import sync_playwright

def run(playwright):
    browser = playwright.chromium.launch()
    page = browser.new_page()
    page.goto('https://example.com')

    # Execute simple JavaScript
    js_result = page.evaluate("() => document.title")
    print(f'Title of the page: {js_result}')

    browser.close()

with sync_playwright() as playwright:
    run(playwright)

In this code snippet, we navigate to 'https://example.com' and execute JavaScript to fetch the title of the webpage. The result is printed in the console, demonstrating how you can retrieve data from the page.

Interacting with Web Elements using JavaScript

Executing more complex JavaScript that interacts with web page elements can also be done in a similar manner. For instance, if you want to click a button on the webpage using JavaScript:

from playwright.sync_api import sync_playwright

def run(playwright):
    browser = playwright.chromium.launch(headless=False)  # headless=False to see the browser actions
    page = browser.new_page()
    page.goto('https://example.com')

    # Assume there is a button with the id 'my-button'
    page.evaluate("() => document.getElementById('my-button').click()")

    browser.close()

with sync_playwright() as playwright:
    run(playwright)

In this script, Playwright loads the page and executes JavaScript to click a button with the ID 'my-button'. This demonstrates integrating Playwright and JavaScript to manipulate web content dynamically.

Using Async Functions

For those preferring asynchronous execution, you can leverage async features as demonstrated below:

import asyncio
from playwright.async_api import async_playwright

async def run(playwright):
    browser = await playwright.chromium.launch()
    page = await browser.new_page()
    await page.goto('https://example.com')

    # Execute async JavaScript
    js_result = await page.evaluate("() => document.title")
    print(f'Title of the page: {js_result}')

    await browser.close()

async def main():
    async with async_playwright() as playwright:
        await run(playwright)

asyncio.run(main())

This async version includes the same functionality but utilizes Python's asyncio library for non-blocking I/O operations.

Conclusion

In this article, we explored how to execute JavaScript in a browser with Playwright using Python. We reviewed basic concepts such as executing simple JavaScript scripts, interacting with web page elements via JavaScript, and setting up both synchronous and asynchronous Playwright scripts. With this knowledge, you can perform complex automation tasks requiring direct interactions with web page scripts.