master
Clemens Klug 2018-05-01 15:12:53 +02:00
parent b1d785b5f4
commit fe533ba321
2 changed files with 86 additions and 9 deletions

View File

@ -16,26 +16,92 @@ Wait, what did I want to do again?
\section{Evaluating Kibana}
%TODO
\subsection{Evaluating Kibana}
To evaluate whether Kibana is a viable approach for the given requirements, I have created a test environment.
This setup is documented in \autoref{app:kibana}.
Two sample datasets were loaded into the Elasticsearch container through HTTP POST requests: \texttt{curl -H 'Content-Type: application/x-ndjson' -XPOST 'elastic:9200/\_bulk?pretty' --data-binary @gamelog.json}.
Once Kibana was told which fields hold the spatial information, it is possible to have a first visualization.
However, this view is optimized for the context of web log processing, so it has a rather low spatial resolution as shown in \autoref{img:kibana} and \autoref{img:kibana2}.
Additionally, the query language restricts the possible research questions the solution can resolve.
This means only the questions expressable in the query language can be answered.
Additionally, this requires the users to master the query language before any resonable conclusions can be extracted.
By building a custom plugin, extension, or modified version, it is possible to circumvent this obstacle.
However, the fast-paced environment of the industry either requires a constant effort of keeping pace, or results in an outdated system rather quickly.\footnote{E.g. the next major release Kibana v6.0.0\footnote{\url{https://github.com/elastic/kibana/releases/tag/v6.0.0}} was released about a year after Kibana v5.0.0\footnote{\url{https://github.com/elastic/kibana/releases/tag/v5.0.0}}. However, the previous major version seems to receive updates for about an year, too.}
\subsection{Conclusion}
After all, the monitoring solutions are no perfect match for this special use case.
The privacy concerns vital in web monitoring prohibit detailed spatial analyzes, the query languages can restrict some questions, and custom extensions require constant integration effort.
Regarding the specified use cases, expecially the non-expert users benefit from a simple to use interface.
The default Kibana worchbench does not qualify for this, a custom interface could improve the situation.
… taugt nich
\begin{itemize}
\item powerful timeseries database
\item complex setup
\item fast paced environment
\item low spatial resolution => privacy optimized
\end{itemize}
\image{\textwidth}{../../PresTeX/images/kibana}{Game trace in Kibana}{img:kibana}
\image{\textwidth}{../../PresTeX/images/kibana2}{Game trace in Kibana}{img:kibana2}
\section{Architecture}
\subsection{Overview}
Based on the learnings above, a custom stack seems like a feasible alternative:
\begin{itemize}
\item Developing from buttom-up takes less time than diving into complex turn-key monitoring solutions.
\item With rather limited amounts of data \footnote{TODO} %TODO
scalable solutions are no hard requirement
\item No core dependecies on fast-paced projects
\item Interfaces tailored on requirements: Simple web interface for non-expert users, CLI and API for researchers with unrestricted possibilities.
\end{itemize}
On the other hand, building such a solution means a lot of infrastructural work to get the project running.
\autoref{img:solution} outlines the three main components of the project.
The already mentioned interfaces (Web and CLI/API) for both target groups are thereby dependent on the third component at the core:
The analysis framework.
\image{\textwidth}{solution.pdf}{Architecture approach}{img:solution}
\subsection{Analysis Framework}
The analysis framework takes game logs, processes their entries, collects results, and renders them to an output.
\autoref{img:flow} shows the data flows through the framework.
Every processed log file has its own analysis chain.
The log entries are fed sequentionally into the analysis chain.
\image{\textwidth}{map-reduce.pdf}{Data flows}{img:flow}
\subsubsection{Analyzer}
An Analyzer takes one log entry at a time.
While processing, the Analyzer can choose to read, manipulate, or consume the log entry.
\paragraph{Reading a log entry}
Every Analyzer can read all of the log entry's contents.
This is obviously the core of the whole framework, as it is the only way to gain knowledge from the log.
Information can be stored in the Analyzer's instance until the log file was processed completely.
\paragraph{Manipulating a log entry}
Every Analyzer can manipulate a log entry.
This can be adding new information, modifying existing information, or deleting information.
\paragraph{Consuming a log entry}
Every Analyzer can consume a log entry.
A consumed log entry is not passed down the analysis chain anymore.
This can be useful to filter verbose logs before computationally expensive operations.
\subsubsection{Result}
\subsubsection{Render}
\subsection{Interfaces}
The interfaces are rather straightforward:
Users select game logs for processing, choose which analyzing methods shall be applied, and finally set a rendering target.
\subsection{old}
Game independance:
\begin{itemize}
\item log importer/transformer necessary
@ -63,5 +129,4 @@ Game independance:
\item API for integration
\item allow load distribution
\end{itemize}
\image{\textwidth}{map-reduce.pdf}{Data flows}{img:flow}
\image{\textwidth}{architecture.pdf}{archoitecure overview}{img:arch}

View File

@ -0,0 +1,12 @@
digraph{
//rankdir="LR";
{
//rank=same;
s [label="Web Interface"];
a [label="Analysis"];
c [label="CLI / API"];
}
s -> a;
c -> a;
}