The customer still has child rows, here in Orders, so SQL Server blocks the delete to keep the data consistent.

Options:
1. Delete the child rows first, then the parent. For related tables work from the lowest level up.
2. Change the constraint to cascade, so deleting a parent removes its children: ALTER TABLE Orders ADD CONSTRAINT FK_Orders_Customers FOREIGN KEY (CustomerId) REFERENCES Customers(CustomerId) ON DELETE CASCADE; (drop the old constraint first). Cascading deletes can remove a lot of data, so only use it when that is really what you want.
3. Mark the customer inactive with a flag instead of deleting. Many systems do this to keep history.

To see what references a table, run EXEC sp_help 'dbo.Customers' and read the Table is referenced by foreign key section.