My textbook says that a function is a set, and that it is a kind of relation, which is also a set.
Now: $$f(x) = x+5$$ is called a function, but the above expression is not a set. This is also true for other functions, like trigonometric functions such as $\sin x$, etc.
I have heard arguments that a function is a rule, and it is expressed as a set. But when we say a function is a kind of relation, this directly implies that it is a set.
So, what is a function?
The neat thing is that you can view it as both!
On one hand, a function can absolutely be thought of as a rule. If you are given a specific $x$ value, then a function like $f(x)=x+5$ tells you where $x$ gets mapped to. So a function can be interpreted as a rule which takes an input, transforms it, and then returns an output.
On the other hand, it can absolutely be thought of as a set. We can write this in set notation, something like $\{(x,x+5) \mid x \in \mathbb{R}\}$, but it’s actually even easier than that. You’ve already seen $f(x)$ as a set, in the form of a line on a graph! That line represents all the points in that same set of pairs written above.
As you go forward in mathematics, you’ll find this is often true, that objects can be viewed through multiple lenses, each of which offers its own insight :)
Good question! We think about functions differently in different contexts.
Even when we're first learning about functions, we learn to represent them in different ways: by a table of values, an equation, or a graph. (Of these, the graph is the one that's clearly a set.) We pick the representation that's most helpful for what we're trying to do, and switching representations mentally can often be a big help in solving problems.
To take your example, in some contexts people will think of $f(x)=x+5$ as a rule that makes numbers bigger by $5$; maybe they'd visualize this as sliding points five units to the right along a number line.
In other situations, people will call $f(x)=x+5$ a line, because they're thinking of the function's graph, which is definitely a set (the set of points $(x,y)$ at which the equation $y=x+5$ is true).
I also want to mention something about the sort of answers I think you'll get to this question. Your specific question naturally stimulates people to ask themselves, "Well, what is a function, really?" Mathematicians are trained to answer this question by referring to the definition of "function." The math community has agreed to define a "function" as a set --- a special kind of relation. That's true even though, when we actually work with functions, we are almost always thinking of them as rules that turn input into output. This is surprising, and it deserves to be explained.
In modern math, "function" is a fundamental concept, and when you think in the context of developing all of mathematics from scratch, the word "function" is going to be defined quite early, at a stage when very few other terms have been defined. In that context, though, we do assume that we already know what sets are. That makes it convenient to start with the "graph" point of view, and define a "function" as a special kind of relation. So beyond a certain point in the curriculum, textbooks are likely to define functions as sets, and people answering this question are very likely to answer in those terms.
By the way, people didn't talk about functions at all until calculus was being developed in the 1600s, and (surprisingly) we didn't really settle on our modern definition of function until well into the 1900s. It was hard to figure out what "function" really ought to mean! So don't let anyone tell you that this is all obvious.
Is function a set or a rule?
It depends on how formal you want to be.
You can define a function $f$ from a set $A$ to a set $B$ as a rule that assigns each element of $A$ to exactly one element of $B$. On the other hand, since nearly all mathematical concepts can be formalized within set theory, you can also be more formal and define a function $f$ from a set $A$ to a set $B$ as a subset of $A\times B$ such that the following two conditions hold:
$(1)$ For each $x\in A$ there is a $y\in B$ such that $(x,y)\in f.$
$(2)$ If $(x,y)\in f$ and $(x,z)\in f,$ then $y=z$.
A function from a set $X$ to a set $Y$ is indeed a special type of relation between $X$ and $Y$, which in turn is a subset of $X \times Y$. Then the notation $f(x)$ is shorthand for "the unique element $y$ of the set $Y$ such that $y$ is related to $x$ by the relation $f$".
The question is then how to specify a function $f: X \to Y$. If $X$ is an infinite set, you can of course hardly specify $f$ by listing all pairs $(x, f(x))$ one by one. Therefore, if you want to actually pick out a specific function, say a function $f: \mathbb{R} \to \mathbb{R}$, you need to provide some description that a human can digest in a finite amount of time. There is no requirement in the definition of a function that every function must have a description digestible by a human in a finite amount of time, i.e. that every function must be definable by some "rule", but that is a different matter.
If you want to be very explicit, instead of talking about "the function $f(x) = x+5$ you can talk about "the unique function $f: \mathbb{R} \to \mathbb{R}$ such that $f(x) = x+5$ for each $x \in \mathbb{R}$". Of course, this kind of pedantic precision is not required in most contexts.
Note that "the function $f(x) = x+5$" is the same function as "the function $f(x) = (x + 6) - 1$" or "the function $f(x) = (x^2 + 3x - 10) / (x - 2)$ for $x \neq 2$ and $f(x) = 7$ for $x = 2$". That is, you can have several descriptions of the same function which are equivalent in a non-obvious way, but they still pick out the same function.
There is no definition of "rule" in mathematics. If, by "rule", you mean "algorithm", then "algorithm" has a precise definition, but a function and the program defined by an algorithm are two very different things and should not be confused at all.
Now, $f(x) = x + 5$ is not a function either (if taken seriously, it's more likely to be an equation with an unspecified variable, $x$). However, the set $\{(x, x+5) \ \vert \ x \in \mathbb{R}\}$ is indeed a function, and it might be the thing you wanted to write. You could also have said « I define $f$ to be the function that maps any real number $x$ to $x+5$».
It's more accurate to say that the graph of the function is a relation and a set. If we have a function as a mapping $f:X \to Y$ then its graph will be the set $\Gamma =\{(x,f(x)) :x \in X\}$. Note that $\Gamma \subset X \times Y$ which makes it a relation. However as a graph fully determines its function and a function fully determines its graph we can claim they're really both the same thing.
I wish to emphasize that a function is conceptually not a set. However, in the conventional foundation of mathematics today, every function can be encoded as a set. This distinction is really important if you want to have a precise understanding. An analogy is that the number 13 is conceptually not the decimal string "13" even though it can be encoded that way.
See, we can encode a function $f$ as the set $\{ \ ⟨x,f(x)⟩ : x∈dom(f) \ \}$, but we can also encode it as $\{ \ ⟨f(x),x⟩ : x∈dom(f) \ \}$. Neither of these sets are the function itself. But of course it is extremely convenient to pick one encoding so that we can construct (encoded) functions using tools from Set Theory (such as using Union to glue together infinitely many approximations of a desired function in order to construct it).
Moreover, even the standard encoding involves ordered pairs. Like functions, an ordered pair ⟨x,y⟩ is conceptually not a set, but can be encoded as $\{ \{x\},\{x,y\}\}$. This encoding is not the only way, obviously. We could have encoded it as $\{ \{y\},\{x,y\}\}$ or $\{ \{0,\{x\}\},\{1,\{y\}\}\}$ or infinitely many other viable encodings. But we have to pick one and say "let's use this encoding for all our ordered pairs" so that we can manipulate ordered pairs within our chosen foundation of mathematics.
See this post for other objects that are not sets conceptually but that one may choose to encode them as sets.
In my view, it's most useful to think of a function as some procedure which takes an input value of some "domain" type, and outputs a value of some "codomain" type.
Now, in practice, in particular in applied fields, you'll often see functions given in a table format. So, for example, a table like the one below could be used to represent data of the number of births per year in some hypothetical city.
Year Number of Births
------------------------
1998 20145
1999 21350
2000 21048
2001 22484
2002 22537
From such a table, you can build a function. Namely, the procedure associated to the table is: you search for the input value in the first column, then move to the second column in the same row, and give that as the output value. Of course, for such a table to give a function, there has to be exactly one row to be found for each possible input value in the domain type. So in the above example, the table corresponds to a function $f$ where for example $f(1999) = 21350$ and $f(2002) = 22537$.
In mathematical contexts, however, we start to run into problems with the table representation when we want to consider functions with an infinite domain: in such a context, we will always run out of paper before being able to list all possible rows of the table. That can be worked around by considering some abstract set of ordered pairs to be a generalization of the table idea. However, since we've now moved to an abstract set instead of something that you can look at in its entirety, my opinion is that this becomes less useful as a mental model. (So, in this point of view, the main utility is primarily in formal contexts, for example if you're starting with just the language and axioms of ZFC and want to define functions in terms of the language of ZFC. That would be something best left for more advanced classes.
Another thing to possibly mention, more relevant in the past than now, would be the log tables, and tables of trigonometric functions, that used to be published. The idea there would be to approximate either the logarithm function, or one of the trigonometric functions, by the function whose rule is given by looking up the nearest values in a large set of samples to the input, and then performing linear interpolation between the two nearest samples. Technically, we might say this also has a codomain type of some notion of "fixed-precision" numbers, since the tables can only give a finite number of decimal digits of the output.)
(As I hinted above, I also tend to like the idea of using informal type theory as a basis for teaching mathematics, as opposed to the idea of using informal set theory which seems to dominate currently. In my opinion, type theory more closely represents the way mathematicians actually think about things on a day-to-day basis, with the possible exception of axiomatic set theorists. I do admit, however, that I have no knowledge of whether formal study has been made of the idea, and whether it would actually result in improved educational outcomes.)
When I learned the basis of set theory, I learned (more or less) what a set was. Then I learned what was a relation between to sets (say $X$ and $Y$): a subset of the cartesian product $X \times Y$.
Then a function is a particular relation where at most one element from $Y$ is related to an element from $X$. As the image element is unique when it exists, we can write $y = f(x)$ in place of $xRy$.
One step further, we found applications, where every element from $X$ has one and only one image. In functional terms, it is a function where the definition set is $X$ and not a proper subset. And even further injections, subjections and bijections.
Of course in common (mathematical) life we forget that functions are nothing but relations because of the numerous operations we use them for, and which would be non sense for relations: composition, derivation, integration etc.
Said differently, functions are defined as relations (and indirectly as sets) and used as rules.
When we say $f: X \to Y$ is a function, we simply mean that for each $x\in X$, the symbol $f(x)$ refers to some particular element of $Y$. It is just a way for us to identify some elements of one set ($Y$, the codomain) by the elements of another ($X$, the domain). In this sense, $f$ isn't really any thing, it's just part of a pattern of symbols used to refer to some things.
This kind of abstract, disconnected idea of a function is both hard to teach with and hard to be rigorous with, and so we like to frame a function as a specific, concrete thing. However, different situations call for different approaches.
The first time we encounter functions in our education is typically in the context of algebra, where we're interested in mappings that can be written as algebraic expressions involving a variable. For example, $f(x) = x+5$ is a function in the sense of the first paragraph -- take any real number $x$, and $x+5$ is another real number, specifically the one referred to by $f(x)$. But in this context, it's like a machine or a rule, a consistent algorithm for turning $x$'s into $f(x)$'s, so that's how it is usually taught. If we're comfortable thinking of $f$ as a black box, whose internal algorithm we may not know, then this idea of "function as a rule" is equivalent to the original.
On the other hand, when it comes to writing rigorous mathematical proofs, having a truly concrete construction of a function from the existing mathematical objects is almost necessary. This is where the idea of functions as sets comes in. We start by defining a relation between $X$ and $Y$ to simply be a subset of the pairs $X\times Y$, and a function to be a relation where each $x$ appears as the first element of exactly one pair in the relation; the second element of that pair is then called $f(x)$. Such a definition gives a consistent base point from which to write proofs about functions. Most of the time, you don't actually need to go back to the formal set construction, but the fact that it exists is reassurance that the mathematics you're doing has some foundation.
We could come up with any number of other models for functions based on different foundations (or lack thereof), but in practice, we tend to work so far away from the foundations that the model is irrelevant to the work. Whatever gets your brain in the space it needs to be to do correct mathematics is sufficient.
This question touches the very fundamental of logics.
In the commonly accepted construction of mathematics used today (with ZFC set theory as the basis, over which all other objects are defined), a function is a set (as explained in the other answers).
We often use notation that hides this, like $f(x) = x + 5$. Note, how this notation is problematic. We need to define the domain of the function, and for each domain we choose we will, in this view, have a different function (while the formulaic definition is, at face value, the same!)$^1$. But in the end, a rigorous proof about functions must be translatable to a proof in the language of sets. (And as noted as a comment by @lalala to one of the other answers: in this view there are many functions that don't have a representation as a string of symbols from a simple counting argument: There are uncountably many functions, but only countably many strings of symbols.)
However, when doing logic, or working in other axiomatizations of mathematics, this definition of a function may not hold.
In logic, the difference between the views is known as the extensional and intensional definition of an object. The extensional definition of the function is the representation of the function as a relation, while the intensional definition is via the formula (or idea) that defines it. In certain situations those two view points may clash and give different results.
This becomes especially important when studying the fundamentals of mathematics and logic. E.g., when doing lambda calculus (where the objects of the universe are functions represented by formulae), you need to explicitly add a derivation rule called $\eta$-reduction to ensure that functions that take the same values are the same function in the calculus (or you can choose to work in a system where that rule does not hold!).
$^1$ It gets even worse, in common abuse of notation, we define an element of $L^2([0,1])$ that way. The elements of $L^2([0,1])$ are equivalence classes of functions $[0,1] \to \mathbb R$. So the formula suddenly represents a set of functions, that only differ on a null set.
What is a function? This question can be answered at many levels. Here is a perspective from the foundations of mathematics.
Mathematics is practised by humans and, as such, is a conceptual art. But mathematicians want to be sure that their concepts are properly grounded and do not turn out to be contradictory, and so illusory. Thus, mathematics makes models of its concepts in set theory, which minimizes the number of concepts and which is generally considered to be the most elementary and rigorous foundation for mathematics. As an example, we have the finite von Neumann ordinals, which model the natural numbers $0,1,2,3,...$ respectively as $\varnothing, \{\varnothing\},\{\varnothing,\{\varnothing\}\},\{\varnothing, \{\varnothing\},\{\varnothing,\{\varnothing\}\}\},...$.
There is lack of consensus about what a function is, exactly. The first and more common view is that the specification of a function must involve specifying its codomain, which includes its range but may be bigger. The second, less common, view ignores the codomain. The second view is good enough for most areas of mathematics. However, the codomain is important in some fields.
According to the second and simpler view, a function $f$ can be modelled by a set $F$ of ordered pairs. The only restriction on such a set is that, if $(x,y)$ and $(x,z)$ are any two elements of $F$, then $y=z$. The set of first components of the ordered pairs is the domain $A$, while the set of second components is the range. In this view, we can naturally identify the function $f$ with the set $F$ and define it as such.
According to the first (majority) view, we have to “attach” the codomain to the function. One way to do this is to define a function $f:A\to C$ as an ordered pair $(F,C)$, where $F$ is the set defined in the previous (simpler) view, and $C$ is the codomain. Then, to extract the heart of the function, which has all the useful information about it in most cases, we unpick the ordered pair to get at the first component $F$. To avoid this distracting technicality, the somewhat vaguer term “together with” is often used for the attachment: “A function is a set of ordered pairs [as $F$ above] together with its codomain...”.
