CUNY AI Lab
CUNY AI Lab

Curating knowledge collections

Led by Zach Muhlbauer and Meha Gupta

New Media Lab · Room 7388.01
CUNY Graduate Center

https://cuny-ai-lab.github.io/sandbox-series/knowledge

Workshop Roadmap

  1. Composing System Prompts Configure model behavior with system prompts.
  2. Curating Knowledge Collections Organize source documents in knowledge collections.
  3. Configuring Skills and Tools Extend model capabilities with skills and tools.

Workshop Agenda

  • Explore and clone · 15 minutes
  • Select new sources · 25 minutes
  • Create and populate collections · 20 minutes
  • Revise system prompts · 10 minutes
  • Test and revise · 20 minutes

Requires individual access, Sandbox sign-in.

Open Sandbox https://chat.ailab.gc.cuny.edu/

Retrieval-Augmented Generation

A Knowledge collection contains documents a custom model can search.

Retrieval-augmented generation (RAG) combines information retrieval with text generation. It retrieves relevant source passages and adds them to a model’s context alongside your request and system prompt instructions.

Search documents → Retrieve passages → Generate response

RAG Under the Hood

Processing documents

  • Parse Extract usable text from files
  • Chunk Divide text into smaller passages
  • Embed Represent passages numerically so they can be searched by similarity

Responding to requests

  • Retrieve Search for passages relevant to a request
  • Add context Append passages to prompt instructions
  • Generate Produce a response using that context

Explore STEM Adventures

STEM Adventure Games uses source documents to guide a text adventure with numbered choices.

Clone this example, then adapt its sources and instructions for a topic and audience you choose.

https://chat.ailab.gc.cuny.edu/?model=stem-adventure-games

Clone Custom Model

Select Workspace in left sidebar, then Models. Search for STEM Adventure Games. Open ⋯ beside it and select Clone.

Workspace Models filtered to STEM Adventure Games, with white annotations identifying Workspace in left sidebar and Clone for original model.

Name Your Copy

Give your copy a unique name and model ID. Base model, system prompt, and settings carry over when you clone.

Scroll to bottom and select Save & Create.

STEM Adventure Games clone form with white annotations identifying its name, model ID, and base model.

Try Custom Model

Open New Chat. Select model ID on bottom right of message box. Choose STEM Adventure Games.

Ask for four adventure options, choose one, and try its numbered choices. Ask for another menu to see new options.

New chat with original STEM Adventure Games selected. Model selector at bottom right of message box is outlined.

Example Sources

At Sandbox Workshop Collection root, instructions.txt describes how sources guide play. Open Wikipedia to see source files.

Sandbox Workshop Collection root with Wikipedia folder and instructions.txt outlined and labeled Source files and Source directions.

Choose Purpose

What should players learn or explore?

Choose a topic and audience for your adaptation of STEM Adventure Games. Identify a question players could investigate through their choices.

Find Sources

Start with one Wikipedia article about your chosen topic. Add more only as needed.

Choose substantial passages that give players something to investigate through their choices.

Create Your Collection

Select Workspace → Knowledge → Create. Name your collection and describe what your sources cover.

Keep access Private and select Create Knowledge.

Create a knowledge base form with STEM Workshop Sources entered and white outlines identifying name, description, and Create Knowledge.

Add Source Text

Select Add Content → New directory. Name it Wikipedia, select Create, then open Wikipedia.

New directory and Wikipedia directory outlined in Sandbox.

Add Source Guidance

Before testing, add two short sources alongside your Wikipedia material.

  • Secondary source — An explanation or analysis that helps interpret your topic.
  • Heuristic source — Questions or rules of thumb that guide player decisions.

Return to collection root. Use Add Content → Add text content to save relevant passages and source links in separate, clearly named files.

Add Instructions

Select collection name above directory contents to return to its root. Select Add Content → Add text content.

Enter instructions as title; Sandbox adds .txt. Use your page titles and filenames to explain which material grounds scenes, which helps interpret them, and which guides player choices. Select Save.

Add text content editor titled instructions, with directions for using three STEM Wikipedia sources. Title outlined; Sandbox adds .txt when saved.

Test Custom Model

  1. Select New Chat in left sidebar.
  2. Select model ID on bottom right of message box. Choose your saved copy.
  3. Ask for four adventure options about your topic. Choose one to begin.

Check Citations

In same chat, ask “Which passage supports this scene? Quote it and cite your source.”

Select a citation beside a response. Does its source passage support that response?

STEM Adventure Games chat with citation beside a response outlined and labeled. Message box remains visible.

Compare Responses

Keep your test chat open. Open STEM Adventure Games in another tab and send same opening request.

https://chat.ailab.gc.cuny.edu/?model=stem-adventure-games

  • How do responses differ?
  • Does your copy use new sources as instructed?

Revise and Retest

Choose one problem you observed. Revise source guidance or system prompt instructions. Add another Wikipedia article only if needed.

For source changes, update copied excerpts and instructions.txt. Wait for processing. For prompt changes, select Save & Update.

Open New Chat, select your saved copy, and repeat your request. Did this change resolve that problem?

Share Custom Model

In your copy’s editor, open Access and grant intended users or groups Read access. Give those same users Read access to its Knowledge collection.

Ask someone to try your copy and check its citations.

Parting Questions

How else could you see yourself using knowledge collections in the future?

What are the limitations of this approach?

Workshop Resources

Check responses against sources in your collection. Next workshop introduces tools that retrieve Wikipedia content live.

Next Workshop

2 3Configuring skills and tools — reveal workshop link

Open workshop

Logistic Map · Petrarch / Creative Clawing

Workshop outline

Workshop Resources

Full workshop copy

Model Registryhttps://ailab.gc.cuny.edu/models/

Session linkhttps://cuny-ai-lab.github.io/sandbox-series/knowledge