Using AI in GitHub Pull Requests

We've been talking for a while about how we can be better at informing the Street Art Cities community about new features and changes to our platform.

Our main way of sharing this info is through our forum, but sometimes it's hard to remember to share an update about a small change.

To encourage ourselves to do so, I built a little bot that will automatically create a draft forum post for every GitHub PR that gets opened, with a simple link to continue editing and post it.

To implement this, I relied on GitHub Models and some JavaScript, which gets triggered through a GitHub Actions workflow.

The script

I'm writing some JavaScript that will be executed by the actions/github-script action. It has access to an already authorized version an the GitHub API client, making it very easy to do the API calls we need to do.

First, we get the full details of the Pull Request:

const { data: pullRequest } = await github.rest.pulls.get({
	owner: context.repo.owner,
	repo: context.repo.repo,
	pull_number: context.issue.number,
});

We can then use those to form our very rudimentary prompt (shortened slightly for clarity) and run it through the GitHub Models API:

// Build our prompt with the PR data
const prompt = `
You are a GitHub bot that helps developers write forum posts based on their pull request descriptions.
The forum post will be a public announcement of the new feature or change, for a non-technical audience.
The platform this is for is called Street Art Cities, and the forum is located at https://streetart.community.
Use informal, simple and friendly language, and make it sound exciting. Our focus is on community, street art and culture.

The pull request title is: "${pullRequest.title}"
The pull request body is: "${pullRequest.body || "No description provided."}"

Output a JSON object with the following fields:
- title: The title of the forum post
- body: The body of the forum post, including a short description of the feature or change, and any relevant links or images. You can use Markdown syntax for formatting.
`;

// Do the API call to GPT-4.1
const { data: llmResponse } = await github.request({
	method: "POST",
	url: "https://models.github.ai/inference/chat/completions",
	data: {
		model: "openai/gpt-4.1",
		messages: [
			{
				role: "user",
				content: prompt,
			},
		],
	},
});

// Extract the JSON object from the response
const suggestion = JSON.parse(llmResponse.choices[0].message.content);

The only thing left is to add our comment to the Pull Request thread:

// Set the content for our body
const commentBody = `
💡 Suggested forum post for this feature update:
	
> **${suggestion.title}**
> ${suggestion.body}
`;

// Get existing comments on the PR
const comments = await github.rest.issues.listComments({
	owner: context.repo.owner,
	repo: context.repo.repo,
	issue_number: context.issue.number,
});

// Check if the bot already posted a comment
const existingComment = comments.data.find((comment) =>
	comment.body.includes("Suggested forum post"),
);

// If not, create a new comment, otherwise update the existing one
if (!existingComment) {
	await github.rest.issues.createComment({
		owner: context.repo.owner,
		repo: context.repo.repo,
		issue_number: context.issue.number,
		body: commentBody,
	});
} else {
	await github.rest.issues.updateComment({
		owner: context.repo.owner,
		repo: context.repo.repo,
		comment_id: existingComment.id,
		body: commentBody,
	});
}

Github Actions workflow

To get the script to run at the right moment, with the right creds, we can set up a GitHub Actions workflow that looks like this:

name: PR Comment - Forum Suggestion

on:
  pull_request:
    types: [opened, edited]

jobs:
  forum-suggestion:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
      models: read
    steps:
      - name: Comment on PR
        uses: actions/github-script@v7
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          script: |
            // The JS script from above

What I especially love about this method is that there's no need to do anything in terms of configuring API credentials at any point, it's all just built-in!