In Python, a token is the smallest individual unit in a program. Just as English sentences are made up of words and punctuation marks, a Python program is made up of tokens.
The Python interpreter breaks down the code into these tokens before executing it. There are 5 main types of tokens in Python:
Keywords are reserved words in Python that have a special meaning. You cannot use them as variable names. Examples include if, else, for, while, def, return, True, False.
Input:
# 'True' and 'and' are keywords
is_active = True
if is_active and True:
print("Keywords in action!")Output:
Keywords in action!
Identifiers are the names given to variables, functions, classes, and other objects by the programmer.
Input:
# 'user_age' and 'name' are identifiers we created
user_age = 25
name = "Alice"
print(name, "is", user_age)Output:
Alice is 25
Literals are the raw, constant data values assigned to variables. They can be numbers, strings, booleans, etc.
Input:
x = 100 # 100 is an Integer Literal
y = "Hello" # "Hello" is a String Literal
z = 10.5 # 10.5 is a Float Literal
print(x, y, z)Output:
100 Hello 10.5
Operators are symbols used to perform operations on variables and values (e.g., +, -, =, ==).
Input:
# The '+' and '=' are operator tokens
result = 10 + 5
print(result)Output:
15
Punctuators are symbols used to organize code structures, like parentheses (), brackets [], braces {}, commas ,, and colons :.
Input:
# The '[]', ',', and ':' are punctuator tokens
my_list = [1, 2, 3]
for item in my_list:
print(item)Output:
1
2
3