Skip to content

refactor: rename github_schema_file_paths table to schema_file_paths #1459

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 64 additions & 35 deletions .ai-agents/memory-bank/activeContext.md

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions .ai-agents/memory-bank/progress.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
- Schema override generation pipeline that creates and stores schema override suggestions based on PR reviews, with context-aware prompts that include both existing schema override and actual schema files content for more accurate suggestions.
- Reusable utility function for fetching schema files content that can be used across different components of the application.
- Type-safe implementation of Supabase queries with proper handling of bigint fields and nested relationships.
- Improved schema file management with the github_schema_file_paths table (renamed from watch_schema_file_patterns) and direct path comparison instead of pattern matching.
- Improved schema file management with the schema_file_paths table (renamed from watch_schema_file_patterns) and direct path comparison instead of pattern matching.
- Standardized Supabase client usage across the codebase using a shared createClient function.
- Enhanced review generation with PR context, incorporating pull request descriptions and comments for more comprehensive analysis.
- Improved naming consistency throughout the codebase, with `fileChanges` replacing `schemaChanges` for better clarity.
Expand Down Expand Up @@ -83,7 +83,7 @@ As part of the transition to Supabase JS, manual rollback processing has been re

The testing approach uses a direct testing strategy with Supabase. We create real records in the database, run the actual functions with these records, and then clean up the test data afterwards. This approach provides more realistic tests that verify the actual functions with real database interactions, leveraging Supabase's ability to be executed directly in test environments.

The schema file management has been improved by renaming the `watch_schema_file_patterns` table to `github_schema_file_paths` and changing from pattern matching to direct path comparison. This provides a more precise and efficient approach to schema file management, aligning with the existing `github_doc_file_paths` model and providing a more consistent approach to file path handling across the application. The implementation includes a migration file that handles the table rename and data transfer, updates to all affected components to use the new table name and field names, and standardization of Supabase client usage across the codebase.
The schema file management has been improved by renaming the `watch_schema_file_patterns` table to `schema_file_paths` and changing from pattern matching to direct path comparison. This provides a more precise and efficient approach to schema file management, aligning with the existing `github_doc_file_paths` model and providing a more consistent approach to file path handling across the application. The implementation includes a migration file that handles the table rename and data transfer, updates to all affected components to use the new table name and field names, and standardization of Supabase client usage across the codebase.

A standardized Supabase migration workflow has been documented in `docs/migrationOpsContext.md`. This document provides detailed guidance on the migration workflow, deployment system, key constraints, and SQL guidelines for creating migrations. Additionally, the `docs/schemaPatterns.md` document defines reusable patterns and rules for database schema design, ensuring consistency across the project. The Supabase migration workflow uses commands like `supabase:migration:new` and `supabase:migration:up` to manage database schema changes.

Expand Down
9 changes: 7 additions & 2 deletions .ai-agents/memory-bank/systemPatterns.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,25 @@
# System Patterns

## System Architecture

Liam Migration is designed to integrate seamlessly with existing development workflows, particularly through its GitHub App integration. It leverages AI components to provide intelligent analysis and suggestions during the migration review process.

The project uses a monorepo structure managed with pnpm workspaces, allowing for maintenance of multiple packages and applications in a single repository while sharing dependencies and code.

## Key Technical Decisions

- **AI-Driven Analysis**: The use of AI to automatically analyze migration impacts, predict risks, and suggest optimizations is central to the product's value proposition.
- **GitHub Integration**: The integration with GitHub repositories allows for automated comments and review approvals, streamlining the development process.
- **OSS and Paid Plan Coexistence**: The product is designed to coexist with its OSS version, offering high-value features in paid plans to ensure a sustainable business model.
- **Monorepo Structure**: The decision to use a monorepo structure with pnpm workspaces enables efficient code sharing and dependency management.
- **TypeScript-First**: All components and functions are written in TypeScript to ensure type safety and improve developer experience.
- **Database Access Strategy**: Using Supabase JS for database access to leverage Supabase's optimized query capabilities and standardize the data access layer. This requires careful handling of type compatibility issues, particularly with bigint fields and nested relationships.
- **Type Safety Approach**: When working with Supabase, we use type assertions carefully to bridge the gap between the Supabase types and our application's expected types, particularly for nested queries and bigint fields.
- **Schema File Management Strategy**: Using direct path comparison instead of pattern matching for schema file detection, with the github_schema_file_paths table (renamed from watch_schema_file_patterns) to store exact file paths.
- **Schema File Management Strategy**: Using direct path comparison instead of pattern matching for schema file detection, with the schema_file_paths table (renamed from watch_schema_file_patterns) to store exact file paths.
- **Standardized Supabase Client Usage**: Using a shared createClient function across the codebase to ensure consistent Supabase client creation and usage.

## Design Patterns

- **Modular Architecture**: The system is built with a modular architecture to allow for easy integration and extension of features.
- **Function Separation**: Business logic is separated into dedicated function files that are called from task definitions, making the code more modular and testable.
- **Task Pipeline**: A series of tasks are chained together to form a complete workflow, with each task responsible for a specific part of the process.
Expand All @@ -28,10 +31,11 @@ The project uses a monorepo structure managed with pnpm workspaces, allowing for
- **Intermediate Mapping Tables**: For many-to-many relationships between entities, we use intermediate mapping tables (e.g., overall_review_knowledge_suggestion_mappings, knowledge_suggestion_doc_mappings) to maintain clean separation of concerns and enable flexible relationship management.
- **Transaction Management**: Moving away from manual rollback processing in server actions to a more robust approach using Supabase RPC for transaction management. This will provide a consistent and reliable way to handle database transactions across the application.
- **Direct Path Comparison**: Using direct path comparison instead of pattern matching for schema file detection, providing a more precise and efficient approach to schema file management.
- **Consistent Naming Conventions**: Using consistent naming conventions across the codebase, such as github_schema_file_paths and github_doc_file_paths for GitHub-integrated file management.
- **Consistent Naming Conventions**: Using consistent naming conventions across the codebase, such as schema_file_paths and github_doc_file_paths for GitHub-integrated file management.
- **Context-Enriched AI Prompts**: The AI review generation incorporates multiple sources of context (PR descriptions, comments, documentation, schema files, and code changes) to provide more comprehensive and relevant analysis.

## Component Relationships

- **GitHub Webhook Handler**: Receives webhook events from GitHub, extracts schema changes, and triggers the review process.
- **Task Pipeline**: A series of tasks (savePullRequestTask → generateReviewTask → saveReviewTask → postCommentTask → generateDocsSuggestionTask/generateSchemaOverrideSuggestionTask → createKnowledgeSuggestionTask) that process the schema changes, generate reviews, and create knowledge suggestions.
- **Review Agent**: Works closely with the GitHub App to provide real-time analysis and feedback on migration changes.
Expand All @@ -41,6 +45,7 @@ The project uses a monorepo structure managed with pnpm workspaces, allowing for
- **Schema Override Generator**: Analyzes PR reviews and schema changes to generate schema override enhancements that improve the schema without changing the core schema.

## Repository Structure

The project follows a structured organization with clear separation of concerns:

- **Apps**: Contains the main web applications (app, docs, erd-sample, migration-web)
Expand Down
2 changes: 1 addition & 1 deletion .liam/schema-override.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ overrides:
tables:
- github_repositories
- github_pull_requests
- github_schema_file_paths
- schema_file_paths
- github_doc_file_paths
- github_pull_request_comments
comment: "Tables related to GitHub. All tables in this group should have a GitHub prefix. External tables must not depend on these tables (e.g., no pullRequestId foreign keys in tables outside this group)."
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ describe('checkSchemaChanges', () => {
try {
// Create a test schema path for the project
const { data: schemaPath, error: schemaPathError } = await supabase
.from('github_schema_file_paths')
.from('schema_file_paths')
.insert({
path: 'migrations/2024_update.sql',
project_id: project.id,
Expand Down Expand Up @@ -142,10 +142,7 @@ describe('checkSchemaChanges', () => {
expect(result).toEqual({ shouldContinue: true })

// Clean up the schema path
await supabase
.from('github_schema_file_paths')
.delete()
.eq('id', schemaPath.id)
await supabase.from('schema_file_paths').delete().eq('id', schemaPath.id)
} finally {
// Clean up the project
await supabase.from('projects').delete().eq('id', project.id)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ export const checkSchemaChanges = async (

const supabase = await createClient()
const { data: schemaPath, error } = await supabase
.from('github_schema_file_paths')
.from('schema_file_paths')
.select('path')
.eq('project_id', projectId)
.single()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ async function getMigrationContents(migrationId: string) {
}

const { data: schemaPath, error: pathError } = await supabase
.from('github_schema_file_paths')
.from('schema_file_paths')
.select('path')
.eq('project_id', overallReview.project_id || '')
.single()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ async function getBranchDetails(projectId: string) {
}

const { data: schemaPath, error: schemaPathError } = await supabase
.from('github_schema_file_paths')
.from('schema_file_paths')
.select('path')
.eq('project_id', projectId)
.single()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ async function getGithubSchemaFilePath(projectId: string) {
const projectId_num = projectId
const supabase = await createClient()
const { data: gitHubSchemaFilePath, error } = await supabase
.from('github_schema_file_paths')
.from('schema_file_paths')
.select('*')
.eq('project_id', projectId_num)
.single()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ async function getERDEditorContent({
.single()

const { data: gitHubSchemaFilePath } = await supabase
.from('github_schema_file_paths')
.from('schema_file_paths')
.select('path, format')
.eq('project_id', projectId)
.eq('path', schemaFilePath)
Expand Down
67 changes: 33 additions & 34 deletions frontend/packages/db/schema/schema.sql
Original file line number Diff line number Diff line change
Expand Up @@ -202,19 +202,6 @@ CREATE TABLE IF NOT EXISTS "public"."github_repositories" (
ALTER TABLE "public"."github_repositories" OWNER TO "postgres";


CREATE TABLE IF NOT EXISTS "public"."github_schema_file_paths" (
"id" "uuid" DEFAULT "gen_random_uuid"() NOT NULL,
"path" "text" NOT NULL,
"project_id" "uuid" NOT NULL,
"created_at" timestamp(3) with time zone DEFAULT CURRENT_TIMESTAMP NOT NULL,
"updated_at" timestamp(3) with time zone NOT NULL,
"format" "public"."schema_format_enum" NOT NULL
);


ALTER TABLE "public"."github_schema_file_paths" OWNER TO "postgres";


CREATE TABLE IF NOT EXISTS "public"."invitations" (
"id" "uuid" DEFAULT "gen_random_uuid"() NOT NULL,
"email" "text" NOT NULL,
Expand Down Expand Up @@ -410,6 +397,19 @@ CREATE TABLE IF NOT EXISTS "public"."review_suggestion_snippets" (
ALTER TABLE "public"."review_suggestion_snippets" OWNER TO "postgres";


CREATE TABLE IF NOT EXISTS "public"."schema_file_paths" (
"id" "uuid" DEFAULT "gen_random_uuid"() NOT NULL,
"path" "text" NOT NULL,
"project_id" "uuid" NOT NULL,
"created_at" timestamp(3) with time zone DEFAULT CURRENT_TIMESTAMP NOT NULL,
"updated_at" timestamp(3) with time zone NOT NULL,
"format" "public"."schema_format_enum" NOT NULL
);


ALTER TABLE "public"."schema_file_paths" OWNER TO "postgres";


CREATE TABLE IF NOT EXISTS "public"."users" (
"id" "uuid" NOT NULL,
"name" "text" NOT NULL,
Expand Down Expand Up @@ -445,12 +445,7 @@ ALTER TABLE ONLY "public"."github_repositories"



ALTER TABLE ONLY "public"."github_schema_file_paths"
ADD CONSTRAINT "github_schema_file_path_path_project_id_key" UNIQUE ("path", "project_id");



ALTER TABLE ONLY "public"."github_schema_file_paths"
ALTER TABLE ONLY "public"."schema_file_paths"
ADD CONSTRAINT "github_schema_file_path_pkey" PRIMARY KEY ("id");


Expand Down Expand Up @@ -567,10 +562,6 @@ CREATE UNIQUE INDEX "github_repository_owner_name_key" ON "public"."github_repos



CREATE UNIQUE INDEX "github_schema_file_path_project_id_key" ON "public"."github_schema_file_paths" USING "btree" ("project_id");



CREATE INDEX "idx_project_organization_id" ON "public"."projects" USING "btree" ("organization_id");


Expand Down Expand Up @@ -607,6 +598,14 @@ CREATE UNIQUE INDEX "project_repository_mapping_project_id_repository_id_key" ON



CREATE UNIQUE INDEX "schema_file_path_path_project_id_key" ON "public"."schema_file_paths" USING "btree" ("path", "project_id");



CREATE UNIQUE INDEX "schema_file_path_project_id_key" ON "public"."schema_file_paths" USING "btree" ("project_id");



ALTER TABLE ONLY "public"."github_doc_file_paths"
ADD CONSTRAINT "github_doc_file_path_project_id_fkey" FOREIGN KEY ("project_id") REFERENCES "public"."projects"("id") ON UPDATE CASCADE ON DELETE RESTRICT;

Expand All @@ -627,11 +626,6 @@ ALTER TABLE ONLY "public"."github_repositories"



ALTER TABLE ONLY "public"."github_schema_file_paths"
ADD CONSTRAINT "github_schema_file_path_project_id_fkey" FOREIGN KEY ("project_id") REFERENCES "public"."projects"("id") ON UPDATE CASCADE ON DELETE RESTRICT;



ALTER TABLE ONLY "public"."invitations"
ADD CONSTRAINT "invitations_invite_by_user_id_fkey" FOREIGN KEY ("invite_by_user_id") REFERENCES "public"."users"("id") ON DELETE CASCADE;

Expand Down Expand Up @@ -747,6 +741,11 @@ ALTER TABLE ONLY "public"."review_suggestion_snippets"



ALTER TABLE ONLY "public"."schema_file_paths"
ADD CONSTRAINT "schema_file_path_project_id_fkey" FOREIGN KEY ("project_id") REFERENCES "public"."projects"("id") ON UPDATE CASCADE ON DELETE RESTRICT;



CREATE POLICY "authenticated_users_can_delete_org_projects" ON "public"."projects" FOR DELETE TO "authenticated" USING (("organization_id" IN ( SELECT "organization_members"."organization_id"
FROM "public"."organization_members"
WHERE ("organization_members"."user_id" = "auth"."uid"()))));
Expand Down Expand Up @@ -1074,12 +1073,6 @@ GRANT ALL ON TABLE "public"."github_repositories" TO "service_role";



GRANT ALL ON TABLE "public"."github_schema_file_paths" TO "anon";
GRANT ALL ON TABLE "public"."github_schema_file_paths" TO "authenticated";
GRANT ALL ON TABLE "public"."github_schema_file_paths" TO "service_role";



GRANT ALL ON TABLE "public"."invitations" TO "anon";
GRANT ALL ON TABLE "public"."invitations" TO "authenticated";
GRANT ALL ON TABLE "public"."invitations" TO "service_role";
Expand Down Expand Up @@ -1170,6 +1163,12 @@ GRANT ALL ON TABLE "public"."review_suggestion_snippets" TO "service_role";



GRANT ALL ON TABLE "public"."schema_file_paths" TO "anon";
GRANT ALL ON TABLE "public"."schema_file_paths" TO "authenticated";
GRANT ALL ON TABLE "public"."schema_file_paths" TO "service_role";



GRANT ALL ON TABLE "public"."users" TO "anon";
GRANT ALL ON TABLE "public"."users" TO "authenticated";
GRANT ALL ON TABLE "public"."users" TO "service_role";
Expand Down
70 changes: 35 additions & 35 deletions frontend/packages/db/supabase/database.types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -174,41 +174,6 @@ export type Database = {
},
]
}
github_schema_file_paths: {
Row: {
created_at: string
format: Database['public']['Enums']['schema_format_enum']
id: string
path: string
project_id: string
updated_at: string
}
Insert: {
created_at?: string
format: Database['public']['Enums']['schema_format_enum']
id?: string
path: string
project_id: string
updated_at: string
}
Update: {
created_at?: string
format?: Database['public']['Enums']['schema_format_enum']
id?: string
path?: string
project_id?: string
updated_at?: string
}
Relationships: [
{
foreignKeyName: 'github_schema_file_path_project_id_fkey'
columns: ['project_id']
isOneToOne: false
referencedRelation: 'projects'
referencedColumns: ['id']
},
]
}
invitations: {
Row: {
email: string
Expand Down Expand Up @@ -789,6 +754,41 @@ export type Database = {
},
]
}
schema_file_paths: {
Row: {
created_at: string
format: Database['public']['Enums']['schema_format_enum']
id: string
path: string
project_id: string
updated_at: string
}
Insert: {
created_at?: string
format: Database['public']['Enums']['schema_format_enum']
id?: string
path: string
project_id: string
updated_at: string
}
Update: {
created_at?: string
format?: Database['public']['Enums']['schema_format_enum']
id?: string
path?: string
project_id?: string
updated_at?: string
}
Relationships: [
{
foreignKeyName: 'schema_file_path_project_id_fkey'
columns: ['project_id']
isOneToOne: false
referencedRelation: 'projects'
referencedColumns: ['id']
},
]
}
users: {
Row: {
email: string
Expand Down
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
BEGIN;

alter table "public"."github_schema_file_paths"
drop constraint if exists "github_schema_file_path_path_project_id_key";

alter table "public"."github_schema_file_paths"
drop constraint if exists "github_schema_file_path_project_id_fkey";

drop index if exists "public"."github_schema_file_path_path_project_id_key";
drop index if exists "public"."github_schema_file_path_project_id_key";

alter table "public"."github_schema_file_paths" rename to "schema_file_paths";

alter table "public"."schema_file_paths" add constraint "schema_file_path_project_id_fkey"
foreign key ("project_id") references "public"."projects"("id")
on update cascade on delete restrict;

create unique index if not exists "schema_file_path_path_project_id_key"
on "public"."schema_file_paths" using btree ("path", "project_id");

create unique index if not exists "schema_file_path_project_id_key"
on "public"."schema_file_paths" using btree ("project_id");

grant all on table "public"."schema_file_paths" to "anon";
grant all on table "public"."schema_file_paths" to "authenticated";
grant all on table "public"."schema_file_paths" to "service_role";

COMMIT;
Loading
Loading