A token is the basic component of Python Script . Tokens are the smallest elements of a program, which are meaningful to the interpreter.
Python Tokens : "A token is the smallest individual unit, or element in the Python program, which is identified by interpreter. They are building blocks of the source code.."

Each logical line in Python is broken down into a series of python tokens, which are basic lexical components. Python converts characters into tokens, each of which corresponds to one of Python’s lexical categories. It is critical to learn and understand its technical jargon, namely Python tokens. Let’s dive in deeper to know about python tokens – keyword, identifier, literal, operator, punctuator in detail.
The diagram shows you different tokens used in Python.
Python interpreter scans written text in the program source code and converts it into tokens during the conversion of source code into machine code.

Identify Tokens in Python:

To identify the tokens in the Python program, let us take an example on it. Look at the following simple program code below.

1. Let us consider the first statement, which consists of nine tokens that are as follows:
a
=
int
(
input
(
“Enter Any Number “
)
)

2. Second Statement consist Tokens as
print
(
“Your Number is=”
,
a
)

Python supports different types of Tokens.

1. Keywords.
2. Constants.
3. Identifiers / variables.
4. Operators.
5. Strings.
6. Delimiters
7. Literals.

1.Keywords: A keyword is a reserved word in a computer language that has a specific meaning. Python keywords form the vocabulary of the python language. Keywords aren’t allowed to be used as identifiers. They are used to define the Python language’s “Syntax” or “Structure.” There are as in all 36 keywords used in Python programming language version 3.11 We can’t use these keywords as an ordinary identifier in a Python program while naming variables, functions, objects, classes, and similar items.
List of Reserved Keyword in Python Language
To list Total number of keywords in python use the following Script

2.Constants:
In programming, a constant refers to the name associated with a value that never changes during the programming execution. Programming constant is different from other constants, and it consists of two things - a name and an associated value. The name will describe what the constant is all about, and the value is the concrete expression of the constant itself. The constants are like variables but their values don’t change during the program execution. Once we define the constant, we can only access its value but cannot change it over time.
Types of Constants:
a. Built in constants:
b. User defined constants

a. Built in constants
1. True and False:
In the official documentation, the True and False are listed as the first constant. These are Python Boolean values and are the instance of the int. A True has a value of 1, and False has a value 0.
2.dunder names :
Python also has many internal dunder names that we can consider constants. There are several of these unique names, we will learn about __name__ and __file__ in this section.
3.Useful String and Math Constants.
There are many valuable constants in the standard library. Some are strictly connected to specific modules, functions, and classes; many are generic, and we can use them in several scenarios. In the below example, we will use the math and string-related modules math and string, respectively.
Let's understand the following example -

b.User-Defined Constants.
To tell other programmers that a given value should be treated as a constant, you must use a widely accepted naming convention for the constant’s identifier or name. Constants are usually defined on a module level and written in all capital letters with underscores separating words.
Examples include MAX_SPEED,PI,GRAVITY etc. as stated in the Constants section of PEP 8.

In python constants are declared using Upper case letters. Hopefully the python programmers must not change the values of constants.

3.Identifiers/Variables:
Just as identity refers to a characteristic that distinguishes a person, the same principle is a python identifier, a token in python. In Python, an identifier is a name given to a Class, Function, or Variable. It aids in distinguishing one entity from others.
Characteristics of Python Identifier:
• The initial letter of the identifier should be any letter or underscore (_).
• Upper and lower case letters have distinct characteristics.
• Except for the initial letter, any digit from 0 to 9 can be part of the identification.
• It shouldn’t be used as a keyword
• Except for the underscore (_), an identifier cannot contain any special characters.
• Identifiers can be as long as you want them to be.
• Case matters when it comes to identifier names. Myself and myself, for example, are not the same thing.

4.Operators:
Operators are tokens that, when applied to variables and other objects in an expression, cause a computation or action to occur. Operands are the variables and objects to which the computation is applied. There are 7 different operators.

i)Arithmetic Operators :

It performs all the mathematical calculations. Here are a few of them:
+     performs Addition . eg. c=a+b
–    Subtract the right-hand operand from the left-hand operand with the subtraction operator. eg. c=a-b
*    operator – Multiplies both sides of the operator’s operands. e.g c=a*b
/    the left-hand operand by the right-hand operand with the division operator. e.g c=a/b
//    the left-hand operand by the right-hand operand with the floor division operator. e.g c=a//b
%    a percentage divides the left-hand operand by the right-hand operand and returns the remainder with the modulus operator.
e.g c=a%b

ii)Relational Operators :

A relational operator is a type of operator that examines the relationship between two operands. Some of the relational operators are:
==    Check if two operands’ values are equal.
!=    Check if two operands’ values are not equal.
>    Check if first operand's value is greater than Second
<    Check if first operand's value is less than Second

iii)Assignment Operators :

The assignment operators are employed to allocate a value to a variable. A few examples are:
+=    It adds the right side input to the left side input and then assigns the result to the left side input.
-=     Augmented assignment operator- It takes the right side operand and subtracts it from the left side operand, then assigns the result to the left side operand.

iv)Logical Operators :

The logical operators compare two boolean expressions and yield a boolean result. Like
The logical AND operator makes a condition true if both operands are true or non-zero.
The logical OR operator returns true if one of the two operands is true or non-zero.
The logical not operator returns true if one of the two operands is false or non-zero.

v)Bitwise Operators :

The bitwise operator manipulates individual bits in one or more bit patterns or binary numbers.
bitwise operators are:
& :    bitwise and
| :    bitwise or
~ :    bitwise not
For example, If a binary XOR operator (^) is set in one input value but not both, it copies the matching binary 1 to the result.

vi)Membership Operators :

The membership operator checks for membership in successions, such as a string, list, or tuple. Like in a membership operator that fetches a variable and if the variable is found in the supplied sequence, evaluate to true; otherwise, evaluate to false.
Membership operators are:
in   :    return true if a sequence with the specified value is present in the object.   e.g x in y
not in :    return true if a sequence with the specified value is not present in the object.   e.g x not in y

vi)Identity Operators :

When comparing the memory locations of two objects, identity operators are used. If two variables point to separate objects, it does not return true; otherwise, it returns false.
Identity Operators:
is  :     Return true if both variables are the same objects.    e.g x is y
is not  :    Return true if both variables are the same objects.     e.g x is not y


5.Strings:
The string is a sequence of characters defined between quotes. (both single and double quotes are applicable to define the string literals.). And these strings perform several operations let us discuss some of them.
Len(String_name): String length
String_name.index(char) : Locate the character in the string
String_name.count(char) : Counts the characters
String_name[::-1] : Reverse the string
String_name.upper() : Converts the strings to upper case
String-name.lower : Coverts the string to lower case

6.Literals:
Literals, tokens in Python, are data elements with a fixed value. Literals return a value for an object of the specified type. Python supports a variety of literals:
• String Literals
• Numeric Literals. These are further of three types, integer, float, and complex literals.
• Boolean Literals
• Literal Collection Lists, tuples, dictionaries, and sets are all examples of literal collections in Python.
• A list is a collection of elements enclosed in square brackets and separated by commas. These variables can be of any data type, and their values can be altered.
• Tuple: A comma-separated list of elements or values in round brackets is also known as a tuple. Values can be of any data type, but they cannot be modified.
• Dictionary: It’s an unsorted collection of key-value pairs.
• The “set” is an unordered collection of objects enclosed in curly braces.

7.Delimeter:
The following tokens serve as delimiters for expressions, lists, dictionaries, and various parts of a statement:

The period (.) is also used in floating-point numbers and in the ellipsis (...) used in extended slicing operations.