Most dialog boxes have a Cancel button, which you can click by pressing esc (Escape) on the keyboard. That’s an easy key to hit as it’s usually at the top left corner of the keyboard.
The “default button” (the one that’s either heavily outlined, or blue) can be triggered with Return or Enter. Enter is the easier key to type thanks to its position at lower right (at least on keyboards with numeric keypads).
The Tab key will take you from place to place in the dialog box. It will highlight every item in a dialog box, including every button, if you turn on Full Keyboard Access in System Preferences/Keyboard and choose the “All controls” option. Trigger a selected button (or checkbox, or radio button) after tabbing to it using the space bar.
Once you’ve tried this you’ll use it all the time.