There is protection implemented to keep the numbers without gaps, by default delete is only allowed on the last created estimate/invoice.
If you don’t need this protection navigate to Setup->Settings->Finance and find:
Delete estimate allowed only on last estimate – Set to NO
Decrement estimate number on delete – Set to NO
These settings can be applied too in case you want to delete only 1 invoice/estimate and after you delete reset the settings to YES again.
Repeat the same steps for invoices if needed.