wörk
parent
b1d785b5f4
commit
fe533ba321
|
|
@ -16,26 +16,92 @@ Wait, what did I want to do again?
|
|||
|
||||
|
||||
\section{Evaluating Kibana}
|
||||
|
||||
%TODO
|
||||
\subsection{Evaluating Kibana}
|
||||
To evaluate whether Kibana is a viable approach for the given requirements, I have created a test environment.
|
||||
This setup is documented in \autoref{app:kibana}.
|
||||
Two sample datasets were loaded into the Elasticsearch container through HTTP POST requests: \texttt{curl -H 'Content-Type: application/x-ndjson' -XPOST 'elastic:9200/\_bulk?pretty' --data-binary @gamelog.json}.
|
||||
Once Kibana was told which fields hold the spatial information, it is possible to have a first visualization.
|
||||
However, this view is optimized for the context of web log processing, so it has a rather low spatial resolution as shown in \autoref{img:kibana} and \autoref{img:kibana2}.
|
||||
|
||||
Additionally, the query language restricts the possible research questions the solution can resolve.
|
||||
This means only the questions expressable in the query language can be answered.
|
||||
Additionally, this requires the users to master the query language before any resonable conclusions can be extracted.
|
||||
|
||||
By building a custom plugin, extension, or modified version, it is possible to circumvent this obstacle.
|
||||
However, the fast-paced environment of the industry either requires a constant effort of keeping pace, or results in an outdated system rather quickly.\footnote{E.g. the next major release Kibana v6.0.0\footnote{\url{https://github.com/elastic/kibana/releases/tag/v6.0.0}} was released about a year after Kibana v5.0.0\footnote{\url{https://github.com/elastic/kibana/releases/tag/v5.0.0}}. However, the previous major version seems to receive updates for about an year, too.}
|
||||
|
||||
\subsection{Conclusion}
|
||||
After all, the monitoring solutions are no perfect match for this special use case.
|
||||
The privacy concerns vital in web monitoring prohibit detailed spatial analyzes, the query languages can restrict some questions, and custom extensions require constant integration effort.
|
||||
|
||||
Regarding the specified use cases, expecially the non-expert users benefit from a simple to use interface.
|
||||
The default Kibana worchbench does not qualify for this, a custom interface could improve the situation.
|
||||
|
||||
… taugt nich
|
||||
\begin{itemize}
|
||||
\item powerful timeseries database
|
||||
\item complex setup
|
||||
\item fast paced environment
|
||||
\item low spatial resolution => privacy optimized
|
||||
\end{itemize}
|
||||
\image{\textwidth}{../../PresTeX/images/kibana}{Game trace in Kibana}{img:kibana}
|
||||
\image{\textwidth}{../../PresTeX/images/kibana2}{Game trace in Kibana}{img:kibana2}
|
||||
|
||||
|
||||
\section{Architecture}
|
||||
|
||||
\subsection{Overview}
|
||||
Based on the learnings above, a custom stack seems like a feasible alternative:
|
||||
\begin{itemize}
|
||||
\item Developing from buttom-up takes less time than diving into complex turn-key monitoring solutions.
|
||||
\item With rather limited amounts of data \footnote{TODO} %TODO
|
||||
scalable solutions are no hard requirement
|
||||
\item No core dependecies on fast-paced projects
|
||||
\item Interfaces tailored on requirements: Simple web interface for non-expert users, CLI and API for researchers with unrestricted possibilities.
|
||||
\end{itemize}
|
||||
|
||||
On the other hand, building such a solution means a lot of infrastructural work to get the project running.
|
||||
|
||||
\autoref{img:solution} outlines the three main components of the project.
|
||||
The already mentioned interfaces (Web and CLI/API) for both target groups are thereby dependent on the third component at the core:
|
||||
The analysis framework.
|
||||
\image{\textwidth}{solution.pdf}{Architecture approach}{img:solution}
|
||||
|
||||
\subsection{Analysis Framework}
|
||||
|
||||
The analysis framework takes game logs, processes their entries, collects results, and renders them to an output.
|
||||
\autoref{img:flow} shows the data flows through the framework.
|
||||
Every processed log file has its own analysis chain.
|
||||
The log entries are fed sequentionally into the analysis chain.
|
||||
\image{\textwidth}{map-reduce.pdf}{Data flows}{img:flow}
|
||||
|
||||
\subsubsection{Analyzer}
|
||||
An Analyzer takes one log entry at a time.
|
||||
While processing, the Analyzer can choose to read, manipulate, or consume the log entry.
|
||||
\paragraph{Reading a log entry}
|
||||
Every Analyzer can read all of the log entry's contents.
|
||||
This is obviously the core of the whole framework, as it is the only way to gain knowledge from the log.
|
||||
Information can be stored in the Analyzer's instance until the log file was processed completely.
|
||||
\paragraph{Manipulating a log entry}
|
||||
Every Analyzer can manipulate a log entry.
|
||||
This can be adding new information, modifying existing information, or deleting information.
|
||||
\paragraph{Consuming a log entry}
|
||||
Every Analyzer can consume a log entry.
|
||||
A consumed log entry is not passed down the analysis chain anymore.
|
||||
This can be useful to filter verbose logs before computationally expensive operations.
|
||||
|
||||
\subsubsection{Result}
|
||||
|
||||
\subsubsection{Render}
|
||||
|
||||
|
||||
\subsection{Interfaces}
|
||||
The interfaces are rather straightforward:
|
||||
Users select game logs for processing, choose which analyzing methods shall be applied, and finally set a rendering target.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
\subsection{old}
|
||||
Game independance:
|
||||
\begin{itemize}
|
||||
\item log importer/transformer necessary
|
||||
|
|
@ -63,5 +129,4 @@ Game independance:
|
|||
\item API for integration
|
||||
\item allow load distribution
|
||||
\end{itemize}
|
||||
\image{\textwidth}{map-reduce.pdf}{Data flows}{img:flow}
|
||||
\image{\textwidth}{architecture.pdf}{archoitecure overview}{img:arch}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,12 @@
|
|||
digraph{
|
||||
//rankdir="LR";
|
||||
{
|
||||
//rank=same;
|
||||
s [label="Web Interface"];
|
||||
a [label="Analysis"];
|
||||
c [label="CLI / API"];
|
||||
}
|
||||
s -> a;
|
||||
c -> a;
|
||||
|
||||
}
|
||||
Loading…
Reference in New Issue