Skip to content

Commit cb5b96c

Browse files
committed
feat(extension): improve page capture, attachments, and browser shortcuts
1 parent 6ff4f32 commit cb5b96c

61 files changed

Lines changed: 3084 additions & 383 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
---
2+
title: Sim browser extension
3+
description: Ask Sim about the page you’re reading, share a screenshot, and open your team’s Sim Links from Chrome.
4+
---
5+
6+
[Install Sim for Chrome](https://chromewebstore.google.com/detail/sim/oanlcahddodnjpdoopcpjlcageejjpmg) to ask questions about a page without leaving it. Open Chrome’s **Extensions** menu and pin **Sim** to keep it one click away.
7+
8+
## Ask about a page
9+
10+
1. Open a web page and click **Sim** in the Chrome toolbar. Sign in to Sim if prompted.
11+
2. In the side panel, select the attachment chip to review the captured text, or use its **×** to remove it. To attach the page again after it changes, use the globe next to the message field.
12+
3. Ask a question and send it. For example, on a release notes page, ask “Which changes affect my team?”
13+
14+
To ask about a passage, select the text on the page, right-click, and choose **Ask Sim about selection**. You can attach up to three snapshots to compare pages. A snapshot stays as it was when you captured it. Chrome settings pages, other extensions, and pages without readable text can’t be attached.
15+
16+
In version 0.1.3 or later, open **+ → Reference current page on send** to include the active tab each time you send a message. This is off by default. The setting appears above the workspace list, and Sim remembers your choice in this Chrome profile. The globe shows **Current page** while this is on. If you attached a page or selection yourself, Sim uses those snapshots instead. Turn the setting off in the same menu.
17+
18+
If Chrome needs access after you switch tabs, your message stays in the composer. Select **Request site access**. If Chrome hasn’t shared the site address with Sim, paste it into **Site address**. Check the site shown on **Allow access**, approve it in Chrome’s permission dialog, then send again. For one-time access, click the Sim toolbar icon on the active tab and retry. Open **+ → Site access** to remove a saved permission.
19+
20+
To share a screenshot, drag the image file onto the open chat. It uploads to Sim and appears as an attachment you can remove before sending. If a website supplies only an image URL, paste the image itself or use the file picker.
21+
22+
For faster access, press **Command + Shift + S** on macOS or **Alt + Shift + S** on Windows and Linux. In version 0.1.3 or later, the shortcut attaches highlighted text when you have a selection. It won't substitute the whole page for an unreadable selection. With no selection, it attaches the page or uses your send-time reference setting. On an already-open sidebar, Chrome may keep keyboard focus on the website. Click the message field, or use Chrome’s pane navigation (on Mac, **Command + Option + Up**).
23+
24+
If Chrome needs access to a new tab, the sidebar shows the shortcut assigned in this Chrome profile, or **Set shortcut** if none is assigned. You can change it under **Extensions → Keyboard shortcuts**.
25+
26+
## Roll out to a team
27+
28+
Share the chat extension’s install link with your team. In managed Chrome, an admin can [allow the extension for an organizational unit](https://support.google.com/chrome/a/answer/6306504) so people can install it themselves.
29+
30+
If your organization has [Sim Links](/search/sim-links), type `sim`, press **Space**, and enter part of a link name for suggestions. To open a shortcut by typing it directly, include the trailing slash, as in `sim/handbook/`. Teams that already use an internal host named `sim` should keep that hostname conflict in mind: the chat extension routes those navigations to Sim.
31+
32+
## Page access and privacy
33+
34+
Sim reads a page or selection when you attach it, or when you send a message with **Reference current page** enabled. For repeat use on one site, choose **+ → Site access** to grant or revoke that site’s permission. Granting access doesn't start background capture. The page text goes to the hosted composer when attached and becomes part of the conversation when you send it.
35+
36+
See [Sim’s browser extension privacy disclosure](https://www.sim.ai/privacy#browser-extension).

‎apps/docs/content/docs/meta.json‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
"./introduction/index",
66
"./getting-started/index",
77
"./desktop/index",
8+
"./browser-extension/index",
89
"---Sim Search---",
910
"search",
1011
"---Build---",
@@ -28,6 +29,7 @@
2829
"defaultOpen": false,
2930
"labels": {
3031
"./desktop/index": "Desktop",
32+
"./browser-extension/index": "Browser extension",
3133
"search": "Overview",
3234
"logs-debugging": "Logs and debugging"
3335
}

‎apps/docs/content/docs/search/sim-links.mdx‎

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,22 +15,34 @@ Names ignore case and punctuation, so `sim/Team-Handbook` and `sim/teamhandbook`
1515

1616
## Open links from the address bar
1717

18-
With the Sim Chrome extension, type `sim/handbook` in the address bar. To find a link, type `sim`, press **Space**, and start typing its name: matching links appear as you type, most opened first. Unlisted links aren't suggested.
18+
In Sim Cloud, open **Sim Links → Browser shortcuts** to install [Sim for Chrome](https://chromewebstore.google.com/detail/sim/oanlcahddodnjpdoopcpjlcageejjpmg) in your work profile. Type `sim` in Chrome’s address bar, press **Space**, and enter part of a link name. Chrome shows matching links, most opened first. Unlisted links can still open by name.
19+
20+
To open links as `sim/handbook`, copy the **Setup address** from that dialog and paste it into Chrome’s address bar once. This opens your Sim Links directory and lets Chrome remember the short host in that profile. If Chrome sends a shortcut to search, enter `sim/handbook/` with a trailing slash. Use **Copy link** in Sim to share an address that also works without the extension.
21+
22+
If `sim` + **Space** sends link names to your search engine even when the address bar shows **Sim**, a Chrome site search may already use the `sim` shortcut. Enter `http://sim/handbook` to open the link directly. To restore suggestions, temporarily turn off the Sim extension, rename the old shortcut in `chrome://settings/searchEngines`, then turn Sim back on. If your organization manages the shortcut, ask your administrator to rename it.
1923

2024
Without the extension, add Sim as a Chrome site search shortcut to type `sim handbook` instead:
2125

2226
1. Open Chrome **Settings → Search engine → Manage search engines and site search**.
2327
2. Next to **Site search**, select **Add**.
24-
3. Enter `Sim` as the name, `sim` as the shortcut, and `https://www.sim.ai/sim/%s` as the URL.
28+
3. Enter `Sim` as the name, `sim` as the shortcut, and `https://www.sim.ai/sim/%s` as the URL. On a self-hosted deployment, use your own Sim origin.
29+
30+
Type `sim`, press **Space**, then type a link name and press **Enter**. For managed profiles without the extension, administrators can distribute the shortcut through Chrome’s [site search policy](https://chromeenterprise.google/policies/site-search-settings/) using `https://www.sim.ai/sim/{searchTerms}` as its URL template.
2531

26-
Type `sim`, press **Space**, then type a link name and press **Enter**. Chrome administrators can add the shortcut for everyone with the `SiteSearchSettings` policy.
32+
### Show favorites in regular Chrome suggestions
33+
34+
In Sim, choose **Copy link** for a favorite. In Chrome’s bookmark manager (`chrome://bookmarks`), add a bookmark with that URL and name it after the shortcut, such as `sim/handbook`. Chrome can suggest that bookmark when you type `sim/handb` or `handbook` in the address bar. Administrators can distribute selected team favorites through Chrome’s [managed bookmarks policy](https://chromeenterprise.google/policies/managed-bookmarks/).
35+
36+
Organization admins can also open **Browser shortcuts → Team bookmarks**, choose listed links, and select **Export bookmarks**. The downloaded JSON contains a folder to merge into Chrome’s managed bookmarks policy. Its links resolve in the exporting organization, so a bookmark shared outside the team cannot open another team's same-name destination.
2737

2838
## Create a link
2939

3040
1. Open **Sim Links** and select **New link**.
3141
2. Enter the **Destination URL** and a **Name**. Names use letters, numbers, and `. : @ & ~ - _`; spaces become hyphens as you type.
3242
3. Optionally add a **Description**, then select **Create**.
3343

44+
With Sim for Chrome 0.1.3 or later, right-click an HTTP or HTTPS link on a page and choose **Create Sim Link**. Check the destination and organization, enter a name, then choose **Create**. Sim strips query parameters and fragments from the initial destination. You can edit the address before saving. Closing the draft creates nothing.
45+
3446
To share a link, right-click it and select **Copy link**.
3547

3648
## Add values to a link

‎apps/extension/DESIGN.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ other workspace resources or browser-control tools.
5656
- Required browser capabilities: side panel, scripting, active tab, context menus,
5757
and short-lived session storage for selection handoff. Required host access is
5858
limited to the configured Sim origin and to `http(s)://sim/*`, which exists only so
59-
one static redirect rule can send a typed `sim/<name>` to the Sim link resolver.
59+
one static redirect rule can send top-level navigations to that host to the Sim link resolver.
6060
- Other HTTP(S) origins are optional grants requested for the current site from a
6161
user gesture. Each grant can be revoked. Keeping the panel open is not a grant
6262
to read every subsequently selected tab.
@@ -68,7 +68,7 @@ other workspace resources or browser-control tools.
6868
configured Sim origin with the signed-in session to suggest matching Sim links,
6969
unless it contains a space (a search) or is longer than a link name can be. Page
7070
content is never sent, and unlisted links are never suggested.
71-
- No always-running page reader. Extraction is tied to an explicit attachment action.
71+
- Page extraction runs on an explicit attachment action or a user-initiated composer Send while Reference current page is enabled. The preference persists locally. Send-time captures are scoped to the request and conversation; navigation, denied access or timeout cancels the send.
7272
- Page bytes are not written to extension local/sync storage. Temporary context-menu
7373
handoff uses trusted-context session storage, expiration and navigation cleanup.
7474
- Sent snapshots become chat data and follow Sim's chat retention and deletion

‎apps/extension/README.md‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,10 @@
22

33
The Chrome extension attaches an explicitly captured page or selection to the existing Sim chat. The native side panel hosts the authenticated `/extension` app surface. Chat opens in the user's organization when available; choosing a workspace is optional. Members without access to Home use the organization's Assistant. Chat rendering, execution, permissions, billing, and history belong to Sim and Mothership.
44

5+
[User and administrator guide](https://docs.sim.ai/browser-extension)
6+
7+
The keyboard shortcut prefers highlighted text and preserves the draft when a selection cannot be read. Chrome’s **Extensions → Keyboard shortcuts** page controls the binding per profile. A link’s context menu opens the existing Sim Links creation form with a reviewed destination; only selecting **Create** writes a link.
8+
59
## Build and load
610

711
From the repository root:
@@ -26,7 +30,7 @@ SIM_EXTENSION_DEV=1 SIM_EXTENSION_ORIGIN=http://localhost:3000 bun run --cwd app
2630
## Data and permissions
2731

2832
- Required capabilities: `activeTab`, `scripting`, `sidePanel`, `storage`, `contextMenus`, `declarativeNetRequestWithHostAccess`, and access to the configured Sim origin and to `http(s)://sim/*`.
29-
- Typing `sim/<name>` in the address bar opens the organization's [Sim link](https://docs.sim.ai/search/sim-links). A single static redirect rule rewrites top-level navigations to `http(s)://sim/*` to `<Sim origin>/sim/*` with the path and query unchanged. The `sim` host permission exists only for that rule: every navigation to that host is redirected before it loads, and no other request is matched or observed. An organization that already serves an intranet host named `sim` should not install this build, since its links would be redirected to Sim.
33+
- Type `sim`, press Space, enter the name, and press Enter to open an organization's [Sim link](https://docs.sim.ai/search/sim-links). Chrome can treat bare `sim/<name>` as a search before the redirect rule runs. Typing `http://sim/<name>` once lets Chrome learn the short host in that profile. A trailing slash (`sim/<name>/`) also selects navigation without relying on that history. A single static redirect rule rewrites top-level navigations to `http(s)://sim/*` to `<Sim origin>/sim/*` with the path and query unchanged. The `sim` host permission exists only for that rule: every navigation to that host is redirected before it loads, and no other request is matched or observed. An organization that already serves an intranet host named `sim` should not install this build, since its links would be redirected to Sim.
3034
- The `sim` address-bar keyword (`sim`, then Space) lists matching Sim links while you type and opens what you enter through the same resolver. It is the extension's only network request: the background worker sends the text typed before the first `/` or `?`, and nothing else, unless that text contains a space (a search) or is longer than a link name can be, to `<Sim origin>/api/sim-links/suggestions` with the signed-in session, and the extension's content security policy allows connections to that origin alone. Unlisted links are never suggested.
3135
- Optional HTTP/HTTPS site access is requested only by a menu action naming that site. Chrome's `activeTab` access is temporary and does not follow arbitrary new sites.
3236
- Page capture runs on demand in Chrome's isolated world. It excludes editable fields, inputs, hidden elements, scripts, and embedded documents. It reads only the top frame. PDF viewers and canvas documents may have no accessible text.
@@ -38,6 +42,8 @@ SIM_EXTENSION_DEV=1 SIM_EXTENSION_ORIGIN=http://localhost:3000 bun run --cwd app
3842
- Captures and queued toolbar intents are scoped to the receiving conversation. Changing conversation, workspace, or account discards an unfinished handoff. Captured attachments remain explicit snapshots when the source tab changes.
3943
- The complete hosted document, including portaled source dialogs and menus, is blocked from session replay. Clipboard writing is delegated only to the configured Sim origin. Clipboard reading and microphone capture are not delegated; voice input remains available in the main app.
4044

45+
With **Reference current page** enabled, a composer Send requests a single fresh snapshot for that conversation. The extension stores only the preference in local storage. A tab change does not read any content. Explicit snapshots take precedence; access denial, navigation and timeouts keep the draft unsent. When Chrome hides the active address, **Site access** lets the user enter a site and confirm its origin in Chrome’s permission dialog. The browser grants the selected scheme, host and port. Uploaded screenshot files stay in the hosted app’s normal attachment flow.
46+
4147
## Verification
4248

4349
```sh

‎apps/extension/package.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "@sim/extension",
3-
"version": "0.1.2",
3+
"version": "0.1.3",
44
"private": true,
55
"type": "module",
66
"license": "Apache-2.0",

‎apps/extension/scripts/build.ts‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ import { fileURLToPath } from 'node:url'
55
import { build } from 'esbuild'
66
import postcss from 'postcss'
77
import loadPostcssConfig from 'postcss-load-config'
8+
import packageJson from '../package.json'
89

910
const root = resolve(dirname(fileURLToPath(import.meta.url)), '..')
1011
const origin = new URL(process.env.SIM_EXTENSION_ORIGIN ?? 'https://www.sim.ai')
@@ -89,8 +90,8 @@ for (const [size, file] of [
8990
await cp(resolve(root, '../sim/public/favicon', file), resolve(dist, `icon-${size}.png`))
9091
}
9192
/**
92-
* Chrome navigates a typed `sim/<name>` to `http://sim/<name>`; this rewrites it to the Sim link
93-
* resolver on the configured origin, keeping the rest of the path and query verbatim.
93+
* Only requests to the short host reach this rule. Chrome can classify bare `sim/<name>` as a
94+
* search before navigation. Preserve the path and query for actual short-host navigations.
9495
*/
9596
const simLinkRules = [
9697
{
@@ -107,7 +108,7 @@ await writeFile(
107108
{
108109
manifest_version: 3,
109110
name: development ? 'Sim (Development)' : 'Sim',
110-
version: '0.1.2',
111+
version: packageJson.version,
111112
description:
112113
"Ask Sim about the page you are reading, and open your organization's sim/ links from the address bar.",
113114
minimum_chrome_version: '120',
@@ -132,7 +133,7 @@ await writeFile(
132133
commands: {
133134
'open-sim': {
134135
suggested_key: { default: 'Alt+Shift+S', mac: 'Command+Shift+S' },
135-
description: 'Open Sim and attach the current page',
136+
description: 'Ask Sim about selected text or the current page',
136137
},
137138
},
138139
content_security_policy: {

0 commit comments

Comments
 (0)