Here's my attempt to the proof:
Since $E$ is a closed set so $E^c=(a_1,b_1) \cup (a_2,b_2) \cup .... \cup (a_k,b_k)...$ (every open set in $\mathbb{R}$ can be written as the countable union of disjoint open intervals).
We define the function $f_i(x)=f(a_i)+\frac{(x-a_i)}{b_i-a_i}(f(b_i)-f(a_i))$ in the interval $(a_i,b_i)$.
We define ,
$g(x)=f(x), x \in E \\ f_i(x), x \in (a_i,b_i) $
We need to show that $g(x)$ is a continuous function . So , we need to show that for every $\epsilon > 0$ there is a $\delta >0$ such that $ |g(x)-g(c)|<\frac{\epsilon}{2}$ , whenever $|x-c| < \frac{\delta}{2}$.
I am taking two cases :
case$1$:if $c \in E^c$,then $c \in (a_i,b_i)$.The function $g(x)$ takes the value $f_i(x)$ in the interval $(a_i,b_i)$ and so there will exist a $\delta1 >0$ for every $\epsilon >0$ such that $ |f_i(x)-f_i(c)|<\epsilon$ , whenever $|x-c| < \delta1$ and $x \in (a_i,b_i)$.Since $c$ is an interior point of the set so we can get $(c-\delta_2,c+\delta_2) \subseteq (a_i,b_i)$, let $\delta = min(\delta_1,\delta_2)$, then $(c-\delta,c+\delta) \subseteq (a_i,b_i)$. $ |g(x)-g(c)|<\epsilon$ , whenever $|x-c| < \delta$.
Case 2: If $c\in E$ for every $\epsilon > 0$ there is a $\delta >0$ such that $ |g(x)-g(c)|<\frac{\epsilon}{2}$ , whenever $|x-c| < \delta$, and $x\in E$.
subcase $1$:if $(c- \delta,c+\delta) \cap E^c =\phi$ then $g(x)$ is continuous at $c$.
subcase $2$:if$(c,c+\delta)$ consists of countable no of points $a_i<a_{i+1}<a_{i+2}<...$ and $(c-\delta,c)$ can consists of points $b_k>b_{k+1}>b_{k+2}>...$.Any other case would not hold true as that would imply $c \in E^c$, which would be a contradiction.
Now let $lim_{x \to a_i^+}(f_i(x))=f(a_i)$[I have been able to show that $a_i \in E$] then we know that by the definition of limits there exists a $\delta_1$ such that $|f_i(x)-f(a_i)|<\frac{\epsilon}{2}$ when $x \in (a_i,a_i+\delta_1)$ , also $lim_{x \to b_k^+}(f_k(x))=f(b_k)$ and by the similar definition we know that there exists a $\delta_2$ such that $|f_k(x)-f(b_k)|<\frac{\epsilon}{2}$ when $x \in (b_k-\delta_2,b_k)$ .Let $d= min\{|a_i + \delta_1-c|,|b_k-\delta_2-c|\}$ then $d>0$.Then in the interval $(c-d,c+d)$ we can see that $|f(x)-f(a_i)|<\frac{\epsilon}{2}...*1*$ and also $|f_i(x)-f(a_i)|<\frac{\epsilon}{2}...*2*$ , now a similar situation arises from $f(b_i)$ thus using the equation 1 and 2 we can conclude that $|g(x) - g(c)|<\epsilon$ in inte interval $(c-d,c+d)$.
subcase$3$ and subcase 4 will consists of cases which have elements $a_i,a_{i+1},...$ in the interval $(c,c+\delta)$ and the other will have elements $b_k,b_{k-1},...$ in the intervals $(c-\delta,c)$.
I know there are many answers to the question but I would be very thankful if someone goes through my one.I was trying to solve Rudin on my own and here's my very humble attemt to question no $5$ of chapter $4$(continuity).
