Result:
For your first integral, I obtained the following simplification:
\begin{align}\color{#FF4F00}{\int^{\frac{\pi}{2}}_0x^3\ln^2(\sin{x})\ {\rm d}x=-\frac{187\pi^6}{26880}-\frac{3}{4}\zeta(\bar{5},1)+\frac{93}{64}\zeta(5)\ln{2}+\frac{3\pi^2}{4}{\rm Li}_4\left(\tfrac{1}{2}\right)-\frac{\pi^4}{64}\ln^2{2}-\frac{3}{8}\zeta^2(3)+\frac{3\pi^2}{8}\zeta(3)\ln{2}+\frac{\pi^2}{32}\ln^4{2}}\end{align}
Derivation:
I will start off with the evaluation of $\displaystyle\int^{\frac{\pi}{2}}_0x^3\ln(\sin{x})\ {\rm d}x$.
\begin{align}
\int^{\frac{\pi}{2}}_0x^3\ln(\sin{x})\ {\rm d}x
&=-\ln{2}\int^{\frac{\pi}{2}}_0x^3\ {\rm d}x-\sum^\infty_{n=1}\frac{1}{n}\int^{\frac{\pi}{2}}_0x^3\cos(2nx)\ {\rm d}x\tag1\\
&=-\frac{\pi^4}{64}\ln{2}+\frac{3\pi^2}{16}\sum^\infty_{n=1}\frac{(-1)^{n-1}}{n^3}-\frac{3}{8}\sum^\infty_{n=1}\frac{\left[1+(-1)^{n-1}\right]}{n^5}\tag2\\
&=-\frac{\pi^4}{64}\ln{2}+\frac{9\pi^2}{64}\zeta(3)-\frac{93}{128}\zeta(5)\tag3\\
\end{align}
Note that
$$\ln^2(\sin{x})={\rm Re}\ln^2(1-e^{i2x})-2\ln{2}\ln(\sin{x})+\left(x-\frac{\pi}{2}\right)^2-\ln^2{2}\tag4$$
Using this identity on the first integral,
\begin{align}
&\small{\int^{\frac{\pi}{2}}_0x^3\ln^2(\sin{x})\ {\rm d}x}\\
=&\small{{\rm Re}\int^{\frac{\pi}{2}}_0x^3\ln^2(1-e^{i2x})\ {\rm d}x+\underbrace{\frac{\pi^6}{3840}+\frac{93}{64}\zeta(5)\ln{2}+\frac{\pi^4}{64}\ln^2{2}-\frac{9\pi^2}{32}\zeta(3)\ln{2}}_{\text{Let this be $\mathcal{K}$}}}\tag5\\
=&\small{\frac{1}{16}{\rm Re}\int^{-1}_1\frac{\ln^3{z}\ln^2(1-z)}{z}{\rm d}z+\mathcal{K}}\tag6\\
=&\small{\frac{1}{16}\underbrace{\int^1_0\frac{\ln^3{z}\ln^2(1+z)}{z}{\rm d}z}_{\mathcal{I}_1}-\frac{3\pi^2}{16}\underbrace{\int^1_0\frac{\ln{z}\ln^2(1+z)}{z}{\rm d}z}_{\mathcal{I}_2}-\frac{1}{16}\underbrace{\int^1_0\frac{\ln^3{z}\ln^2(1-z)}{z}{\rm d}z}_{\mathcal{I}_3}+\mathcal{K}\tag7}
\end{align}
I am unable to find a closed form for $\mathcal{I}_1$ (I doubt there is one but feel free to prove me wrong), so I have no choice but to express it as a multiple zeta value.
\begin{align}
\mathcal{I}_1
&=-\frac{1}{2}\int^1_0\frac{\ln^4{z}\ln(1+z)}{1+z}{\rm d}z\tag8\\
&=-\frac{1}{2}\sum^\infty_{n=1}(-1)^{n+1}H_n\int^1_0z^n\ln^4{z}\ {\rm d}z\tag9\\
&=-12\sum^\infty_{n=1}\frac{(-1)^{n+1}H_n}{(n+1)^5}\\
&=-12\zeta(\bar{5},1)
\end{align}
$\mathcal{I}_2$ can be handled, but its evaluation is very lengthy so I will simply use a previous result derived from another post.
\begin{align}
\mathcal{I}_2
&=-\int^1_0\frac{\ln^2{z}\ln(1+z)}{1+z}{\rm d}z\tag{10}\\
&=\sum^\infty_{n=1}(-1)^{n}H_n\int^1_0z^n\ln^2{z}\ {\rm d}z\tag{11}\\
&=2\sum^\infty_{n=1}\frac{(-1)^{n+1}H_n}{n^3}-2\sum^\infty_{n=1}\frac{(-1)^{n+1}}{n^4}\\
&=-4{\rm Li}_4\left(\tfrac{1}{2}\right)+\frac{\pi^4}{24}-\frac{7}{2}\zeta(3)\ln{2}+\frac{\pi^2}{6}\ln^2{2}-\frac{1}{6}\ln^4{2}\tag{12}\\
\end{align}
$\mathcal{I}_3$ is the easiest of the three integrals.
\begin{align}
\mathcal{I}_3
&=\frac{1}{2}\int^1_0\frac{\ln^4{z}\ln(1-z)}{1-z}{\rm d}z\tag{13}\\
&=-12\sum^\infty_{n=1}\frac{H_n}{(n+1)^5}\tag{14}\\
&=-\frac{\pi^6}{105}+6\zeta^2(3)\tag{15}\\
\end{align}
Combining these results, we obtain
\begin{align}\color{#FF4F00}{\int^{\frac{\pi}{2}}_0x^3\ln^2(\sin{x})\ {\rm d}x=-\frac{187\pi^6}{26880}-\frac{3}{4}\zeta(\bar{5},1)+\frac{93}{64}\zeta(5)\ln{2}+\frac{3\pi^2}{4}{\rm Li}_4\left(\tfrac{1}{2}\right)-\frac{\pi^4}{64}\ln^2{2}-\frac{3}{8}\zeta^2(3)+\frac{3\pi^2}{8}\zeta(3)\ln{2}+\frac{\pi^2}{32}\ln^4{2}}\end{align}
A similar methodology can be used to simplify your other integral. Note that this isn't exactly a closed form due to the multiple zeta value.
Explanation:
$(1): \text{Use the Fourier series expansion of $\ln(\sin{x})$.}$
$(2): \text{Integrate by parts.}$
$(5): \text{Apply $(3)$ and $(4)$.}$
$(6): \text{Let $z=e^{i2x}$.}$
$(7): \text{Extract the real part.}$
$(8): \text{Integrate by parts.}$
$(9): \text{Expand using the generating function of $H_n$.}$
$(10): \text{Integrate by parts.}$
$(11): \text{Expand using the generating function of $H_n$.}$
$(12): \text{See}$ here.
$(13): \text{Integrate by parts.}$
$(14): \text{Expand using the generating function of $H_n$.}$
$(15): \text{See}$ here.
