A collection's icon comes straight from its Notion page icon. Open the collection page, hover over the title, and click Add icon. Pick an emoji — or switch to the Upload tab for a custom image. A square PNG with a transparent background looks best on the site.

After the next sync, the icon shows up everywhere the collection does: on the category cards in the Help Center layout, in the top tabs of the Documentation layout, and in search results. Want a different one later? Click the icon in Notion, choose a replacement, and resync.
