Infinitive is the basic form of the verb, this doesn't have  reference of any tense. Infinitive is always used with to, for instance: to speak, to travel, to listen. 

INFINITIVE: verb form that can be used as: noun, adjective or an adverb.

• INFINITIVES USED AS A NOUNS: To fly was an ambition for humans. (Flying or to fly, both are correct)
• DIRECT OBJECT: They start to sink  (direct object of start)
• INFINITIVES USED AS AN ADJETCTIVES: The one to ask is your plumber (adjective modifying the pronoun one)
• It's important to take into account the list of verbs that are followed with infinitive, for example: Try, forget, need and so on.