Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/scripts/http-e2e.sh
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,7 @@ case "$group" in
export REDIS_URL=redis://127.0.0.1:6379
export NEXT_PUBLIC_FORCE_HOSTED=false
export KNOWLEDGE_MEMBER_ACCESS=true CREDENTIAL_GROUPS=true ACCESS_CONTROL_ENABLED=true
export SIM_LINKS=true
export NEXT_PUBLIC_ACCESS_CONTROL_ENABLED=true
export API_ENCRYPTION_KEY="$ENCRYPTION_KEY"
start_browser_install "$group" --no-shell chromium
Expand Down
44 changes: 27 additions & 17 deletions apps/docs/content/docs/browser-extension/index.mdx
Original file line number Diff line number Diff line change
@@ -1,38 +1,48 @@
---
title: Sim browser extension
description: Ask Sim about the page you’re reading, share a screenshot, and open your team’s go/ links from Chrome.
description: Ask Sim about a page, dictate a message, and open your team’s go/ links from Chrome.
---

[Install Sim for Chrome](https://chromewebstore.google.com/detail/sim/oanlcahddodnjpdoopcpjlcageejjpmg) to ask questions about a page without leaving it. Open Chrome’s **Extensions** menu and pin **Sim** to keep it one click away.
[Install Sim for Chrome](https://chromewebstore.google.com/detail/sim/oanlcahddodnjpdoopcpjlcageejjpmg), then pin **Sim** in Chrome’s **Extensions** menu.

## Ask about a page

1. Open a web page and click **Sim** in the Chrome toolbar. Sign in to Sim if prompted.
2. In the side panel, select the attachment chip to review the captured text, or use its **×** to remove it. To attach the page again after it changes, use the globe next to the message field.
3. Ask a question and send it. For example, on a release notes page, ask “Which changes affect my team?”
1. Open a page and click **Sim** in the toolbar. Sign in if prompted.
2. Review the attached page by clicking its chip. Use **×** to remove it.
3. Type your question and send it.

To ask about a passage, select the text on the page, right-click, and choose **Ask Sim about selection**. You can attach up to three snapshots to compare pages. A snapshot stays as it was when you captured it. Chrome settings pages, other extensions, and pages without readable text can’t be attached.
To ask about a passage, select it, right-click, and choose **Ask Sim about selection**. To add another page, open it and use **+ → Current page**. You can attach up to three snapshots; each stays as it was when captured.

Open **+ → Reference current page on send** to include the active tab each time you send a message. This is off by default. The setting appears above the workspace list, and Sim remembers your choice in this Chrome profile. The globe shows **Current page** while this is on. If you attached a page or selection yourself, Sim uses those snapshots instead. Turn the setting off in the same menu.
Choose **+ → Page access → Allow across websites** and approve Chrome’s prompt once to use pages across websites. Your draft stays unsent during setup. To change access later, choose **Page access → Manage in browser**.

If Chrome needs access after you switch tabs, your message stays in the composer. Select **Request site access**. If Chrome hasn’t shared the site address with Sim, paste it into **Site address**. Check the site shown on **Allow access**, approve it in Chrome’s permission dialog, then send again. For one-time access, click the Sim toolbar icon on the active tab and retry. Open **+ → Site access** to remove a saved permission.
Turn on **+ → Reference current page on send** to include the active page whenever you send. Sim remembers your choice. Manually attached pages or selections take priority.

To share a screenshot, drag the image file onto the open chat. It uploads to Sim and appears as an attachment you can remove before sending. If a website supplies only an image URL, paste the image itself or use the file picker.
Chrome settings, other extensions, and pages without readable text can’t be attached.

For faster access, press **Command + Shift + S** on macOS or **Alt + Shift + S** on Windows and Linux. The shortcut attaches highlighted text when you have a selection. It won't substitute the whole page for an unreadable selection. With no selection, it attaches the page or uses your send-time reference setting. On an already-open sidebar, Chrome may keep keyboard focus on the website. Click the message field, or use Chrome’s pane navigation (on Mac, **Command + Option + Up**).
## Dictate or attach a screenshot

If Chrome needs access to a new tab, the sidebar shows the shortcut assigned in this Chrome profile, or **Set shortcut** if none is assigned. You can change it under **Extensions → Keyboard shortcuts**.
Click the microphone beside Send to dictate. Allow microphone access if Chrome asks, then click again to stop. Edit the draft and send when ready. If access is blocked, choose **Open settings**.

## Roll out to a team
Drag a screenshot file onto Chat, or use the file picker. It uploads as an attachment you can remove before sending. Dragging an image URL from a website doesn’t attach the image itself.

Share the chat extension’s install link with your team. In managed Chrome, an admin can [allow the extension for an organizational unit](https://support.google.com/chrome/a/answer/6306504) so people can install it themselves.
## Keyboard shortcut

If your organization has [Links](/search/sim-links), type `sim`, press **Space**, and enter part of a link name for suggestions. To open a shortcut by typing it directly, include the trailing slash, as in `go/handbook/` or `sim/handbook/`. Teams that already use an internal host named `go` or `sim`, or another go-links extension, should keep that conflict in mind: the extension routes those navigations to Sim, and when two extensions redirect the same address, Chrome uses the most recently installed one.
Press **Command + Shift + S** on macOS or **Alt + Shift + S** on Windows and Linux. Sim attaches your selection, or the current page when **Reference current page** is off. With it on, the page is included when you send. An unreadable selection stays unsent.

**+ → Browser shortcuts** can recover explicit `go/handbook` and `sim/handbook` searches on Google, Bing, DuckDuckGo, and Yahoo. Enable your provider and approve the site access shown by Chrome. This leaves ordinary searches and your default search engine unchanged. See [shortcut setup and supported inputs](/search/sim-links#open-links-from-the-address-bar).
Change the binding in Chrome’s **Extensions → Keyboard shortcuts**. If Chrome keeps focus on the page, click the message field or use its pane-navigation shortcut (**Command + Option + Up** on Mac).

## Page access and privacy
## Open team links

Sim reads a page or selection when you attach it, or when you send a message with **Reference current page** enabled. For repeat use on one site, choose **+ → Site access** to grant or revoke that site’s permission. Granting access doesn't start background capture. The page text goes to the hosted composer when attached and becomes part of the conversation when you send it.
Open **Links → Browser shortcuts** in Sim and select **Enable go/ shortcuts**. Then type a shortcut such as `go/handbook` in Chrome’s address bar. You can enable `sim/handbook` there too.

For suggestions, type `sim`, press **Space**, and enter part of a link name. If Chrome searches instead of opening a shortcut, enable your search engine under the extension’s **+ → Browser shortcuts**. Other searches stay unchanged.

See [Links](/search/sim-links#open-links-from-the-address-bar) for supported inputs, Glean conflicts, and team rollout. Administrators can [allow the extension for a Chrome organizational unit](https://support.google.com/chrome/a/answer/6306504).

## Privacy

Sim reads page text when you attach it or send with **Reference current page** on. Approval alone doesn’t start background capture. Attached text appears in the Sim composer before you send; submitted content follows your conversation’s retention settings.

Voice input sends microphone audio to Sim’s transcription provider while recording. It does not record tab audio or start when you open the panel. The transcript stays in your draft until you send.

See [Sim’s browser extension privacy disclosure](https://www.sim.ai/privacy#browser-extension).
20 changes: 15 additions & 5 deletions apps/docs/content/docs/search/sim-links.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,25 @@ Names ignore case and punctuation, so `sim/Team-Handbook` and `sim/teamhandbook`

## Open links from the address bar

In Sim Cloud, open **Links → Browser shortcuts** to install [Sim for Chrome](https://chromewebstore.google.com/detail/sim/oanlcahddodnjpdoopcpjlcageejjpmg) in your work profile. Type `sim` in Chrome’s address bar, press **Space**, and enter part of a link name. Chrome shows matching links, most opened first. Unlisted links can still open by name.
[Install Sim for Chrome](https://chromewebstore.google.com/detail/sim/oanlcahddodnjpdoopcpjlcageejjpmg) in your work profile. In Sim, open **Links → Browser shortcuts** and select **Enable go/ shortcuts**. Type `go/handbook` in the address bar to open a link. Enable **sim/ shortcuts** there if you prefer `sim/handbook`.

To open links as `go/handbook` or `sim/handbook`, select **Enable go/ shortcuts** and **Enable sim/ shortcuts** in that dialog once each, using the Chrome profile where Sim is installed. Each opens your Links directory in a new tab and lets Chrome remember the short host in that profile. If Chrome still sends a shortcut to search, enter `http://sim/handbook` or `sim/handbook/` with a trailing slash. Use **Copy link** in Sim to share an address that also works without the extension.
For suggestions, type `sim`, press **Space**, and enter part of a name. Matching links appear with the most opened first. Unlisted links still open by name.

Open the extension sidebar’s **+ → Browser shortcuts** and enable your search provider to recover `go/handbook` or `sim/handbook` when Chrome treats it as a search. Approve access to the exact sites shown. This is optional and supports Google (`google.com`), Bing, DuckDuckGo, and Yahoo. Sim remembers the choice in this Chrome profile. Ordinary searches still use your search engine; its default is unchanged.
### If a shortcut opens search

Recovery matches a whole `go/<name>` or `sim/<name>` shortcut, including simple slash-separated values such as `go/jira/123`. For values containing spaces, query parameters, unsupported search domains, or a managed browser that blocks site access, use `sim` + **Space** or the full `https://www.sim.ai/sim/<name>` address. Turn recovery off in **Browser shortcuts**; this keeps the site permission, which you can remove through **Site access** or Chrome’s extension settings.
Open the extension’s **+ → Browser shortcuts** and turn on your search engine. Approve Chrome’s site-access prompt if shown. Google (`google.com`), Bing, DuckDuckGo, and Yahoo are supported. Other searches and your default search engine stay unchanged.

If `sim` + **Space** sends link names to your search engine even when the address bar shows **Sim**, a Chrome site search may already use the `sim` shortcut. Enter `http://sim/handbook` to open the link directly. To restore suggestions, temporarily turn off the Sim extension, rename the old shortcut in `chrome://settings/searchEngines`, then turn Sim back on. If your organization manages the shortcut, ask your administrator to rename it.
This handles whole shortcuts such as `go/handbook` or `go/jira/123`. For spaces, query parameters, unsupported search sites, or blocked site access, use `sim` + **Space** or the full `https://www.sim.ai/sim/<name>` address. A trailing slash (`go/handbook/`) or `http://go/handbook` also tells Chrome to navigate directly.

Turning shortcuts off keeps site access. To remove it, open Chrome’s **Extensions → Manage extensions → Sim → Details**.

### If another tool uses go/ or sim

Sim routes both `go/` and `sim/` to your organization’s links. If Glean or another extension also handles `go/`, Chrome uses the most recently installed extension. Disable the other tool’s go-link handling or remove that extension to use Sim consistently. An existing intranet host named `go` or `sim` conflicts too.

If `sim` + **Space** still searches while the address bar shows **Sim**, an old Chrome site-search shortcut may use the same keyword. Turn off the Sim extension, rename that shortcut in `chrome://settings/searchEngines`, then turn Sim back on. Ask your administrator if the shortcut is managed.

### Use links without the extension

Without the extension, add Sim as a Chrome site search shortcut to type `sim handbook` instead:

Expand Down
20 changes: 13 additions & 7 deletions apps/extension/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,20 @@ conversation persistence, streaming recovery, tool permissions, and organization
or workspace authorization. The panel keeps Fast mode and omits the model selector.
A conversation can continue in the full app.

Voice input uses the existing microphone control beside Send. The browser asks for
microphone permission on first use. Transcription updates the editable draft and
never sends it automatically. Stop ends recording; Send and conversation changes
discard unfinished transcription. The iframe delegates microphone access only to
the configured Sim origin, and blocked access offers the browser's microphone settings.

The user signs in in a normal Sim tab and lands in their organization chat. Choosing
a workspace is optional; members without Build access use Assistant. They can open
or create a conversation and explicitly attach the current page or highlighted text.
The composer’s + menu offers Current page and Selected text. Pending snapshots sit
inside the composer as compact removable chips alongside file attachments. Opening one
shows the captured text, source, capture time, and conversation-retention disclosure.
Partial captures remain labeled in the attachment row. Site access lives in the
composer’s Site access action, which opens an extension-owned dialog; successful delivery adds no persistent status banner.
Partial captures remain labeled in the attachment row. Page access lives in the
composer’s Page access action, which opens an extension-owned dialog; successful delivery adds no persistent status banner.
Snapshots are immutable: navigating to another page cannot silently change an attached
source. Removing an attachment, changing organization or workspace, or starting a new conversation
clears pending page context. Several explicitly attached pages can support comparison.
Expand All @@ -34,7 +40,7 @@ Existing Sim server tools remain subject to their existing authorization and app
## Ownership and data flow

1. `apps/extension`: packaged MV3 service worker, side-panel host, bounded content
extraction, per-site permission controls, build artifacts, browser verification.
extraction, browser-wide page permission setup, build artifacts, browser verification.
2. `packages/browser-extension-protocol`: versioned, strictly validated messages and
immutable page snapshots shared by the extension, app, and server.
3. `apps/sim/app/extension`: session gate, organization landing, optional workspace picker, app providers,
Expand All @@ -57,9 +63,9 @@ other workspace resources or browser-control tools.
and short-lived session storage for selection handoff. Required host access is
limited to the configured Sim origin and to `http(s)://sim/*` and `http(s)://go/*`, which exist
only so one static redirect rule can send top-level navigations to those hosts to the link resolver.
- Other HTTP(S) origins are optional grants requested for the current site from a
user gesture. Each grant can be revoked. Keeping the panel open is not a grant
to read every subsequently selected tab.
- HTTP(S) page access is one optional browser-wide grant, requested from a direct
user gesture. The browser owns revocation and restrictions. Approval enables
capture across sites; switching tabs alone does not extract content.
- No history, bookmarks, cookie API, debugger, native messaging, external messaging,
or arbitrary remote code execution. The only network interception is the static
`sim/` and `go/` redirect rule above, plus opt-in search recovery.
Expand Down Expand Up @@ -126,7 +132,7 @@ An absent backend or account is an unverified gate, not a pass.
- [Chrome activeTab](https://developer.chrome.com/docs/extensions/develop/concepts/activeTab)
and [optional permissions](https://developer.chrome.com/docs/extensions/reference/api/permissions):
temporary access follows user invocation; cross-origin navigation requires fresh
authority. Request persistent access per site only when selected by the user.
authority. Request optional access across HTTP(S) websites once, matching Glean’s all-websites page-context option.
- [Chrome extension security](https://developer.chrome.com/docs/extensions/develop/security-privacy/stay-secure)
and [OWASP extension guidance](https://cheatsheetseries.owasp.org/cheatsheets/Browser_Extension_Vulnerabilities_Cheat_Sheet.html):
minimize privileges and treat all incoming page messages as attacker-controlled.
Expand Down
Loading
Loading