The $? variable in shell scripting stores the exit code of the last executed command. A value of 0 typically indicates success, while non-zero values indicate various error conditions. This is essential for error handling and conditional logic in scripts.