• MikolajFromDataEngineering

    At Fandom Data Engineering team, we are responsible for not only building internal analytics capabilities, but also delivering features such as recommendations and Analytics Dashboard for our community.
    In total, we are custodians of ~170 TB Data Warehouse.

    Internally, we are using Apache Airflow to build and manage ~35 ETLs (Extract-Transform-Load), making it all possible.
    This is a challenging task, no matter what technology is used.
    Based on our experience, here are some questions we consider essential before anyone writes a data pipeline.

    The world of data is alluring to almost anyone in a modern organisation.
    Especially if it can be automated.

    Most of our processes expose their results as tables on Amazon Athena, available for later consumption by to…

    Read more >
  • JacekThePie2

    Is using AWS cloud cool? Definitely! What is not so cool about using those kinds of services is quite a limited way to keep track of how much it costs or avoid surprising bills at the end of the month. Working on-premise does not make you concerned about those matters, but on the other hand, you have to give up on benefits that the cloud serves, like flexibility, auto-scaling, short time-to-market, robustness, easy research, and development, etc. 
    The aforementioned end of the month comes, and after receiving a receipt, you are thinking only about two things: how is it possible that I burned so much money on this and how to make the next receipt less painful. Answer six questions below and find out how much you can save.

    As it may sound obvi…

    Read more >
  • Kviatkovsky

    This blog post is a follow-up of Enhancing XcodeGen for simpler maintenance of dependencies in modular iOS app . In the mentioned article I showed you how we enhanced XcodeGen format so only direct dependencies of modules are needed to be specified in project definitions. In this article I would like to present to you a solution that made our PR checks faster, as we run them only for modules that need it. It’s best if you first read the mentioned article to fully understand the issues that we faced. Anyway, I’ll start with a quick recap of the problem.

    In the sample setup we had 3 projects defined with XcodeGen:

    • "ModularApp" application
    • “FeatureModule” dynamic framework imported by the application
    • “ApiModule” dynamic framework imported by the …

    Read more >
  • IrTriskit

    Beyond Remote

    July 17, 2020 by IrTriskit

    Remote work is quickly becoming the norm in the tech industry, but that was definitely not the case pre-2020. At D&D Beyond, we’ve always had some remote staff, but the vast majority of our team has been collocated at our office in Huntsville, Alabama. Historically, there were two main concerns around embracing more remote work - communication and culture.

    As I mentioned, we were already working with some remote folks, but that certainly had some challenges. Most of our important communication happened in-office and in a lot of cases required you to be in close proximity to the conversation or else you’d probably miss out. Even having a desk on the opposite side of the office presented problems, let alone on the other side of the country or…

    Read more >
  • Kviatkovsky

    Developers that take the challenge of breaking the monolith in their iOS codebase and splitting it into separate modules face another problem: How to best manage multiple .xcodeproj files and potential conflict resolution on them? Fortunately, there’s a really convenient tool called XcodeGen that makes it quite easy – it generates the .xcodeproj files based on the YAML files stored in the codebase. Using YAML files greatly simplifies conflict resolution since you no longer need to version control .xcodeproj files. Instead, you can define your project with a much simpler syntax.

    However, when you have a bunch of dynamic framework targets with dependencies between them and other third party dependencies, like Carthage-built libraries, managin…

    Read more >
  • Andrzej Łukaszewski

    In 2007 Steve Souders wrote a book High Performance Web Sites which was the first book I read about web performance. Two years later he published another book with a catchy title Even Faster Web Sites which I also read and I recommend both of them as a great source of basics in the web performance world.

    However, the first one is really special in my opinion because 10 out of 14 pieces of advice are connected to the frontend. This led to forming the golden rule of web performance which I’d like every web developer to know: “Only 10-20% of the end user response time is spent downloading the HTML document. The other 80-90% is spent downloading all the components in the page”.

    With the golden rule of web performance in mind let’s fast-forward to…

    Read more >
  • Gaperlinski

    Lately, I’ve been spending a lot of my free time on a side project that focuses on converting real-time data obtained from AVCaptureSession into H.264 and AAC streams and saving them to an .mp4 file. While AVFoundation offers developers a pretty straightforward way of converting CMSampleBuffer into the desired format by means of AVAssetWriterInput and relevant output settings, I was unable to use that API because my use case required me to have direct access to converted raw bytes before they got appended to the target file.

    In my case, I needed to utilize VTCompressionSession and AudioConverter in order to encode the data and then pass the resulting CMSampleBuffers to AVAssetWriter. I’m not going to get into the details of compression here; …

    Read more >
  • Damian Bielecki

    Our journey with tree shaking and some nuisance we had to face.

    In Ad Engineering we work on AdEngine our library for managing ads, as well as on integrating that library with various sites (apps) we support. So, we have a library and multiple apps that we need to adjust for tree shaking to work.

    For building library (AdEngine) we use Typescript with Rollup.

    For building apps we use Typescript with Webpack.

    Before tree shaking we used to split AdEngine into separate modules to limit bundle size. That solution was hard to maintain and did not yield expected results. Every module was an autonomous unit and it usually meant including duplicates of code in each of them. Moreover, it was not clear how to split modules for the best efficiency as app…

    Read more >
  • Dorotamierzwa15

    Sroka is an open source Python library that enables quick access to various data sources (e.g. S3, Athena, Google Ad Manager). Name of this library comes from Polish language - “sroka” means magpie.

    Having introduced Sroka to the public in the first Medium article , I want to now share an example of a real analysis, where Sroka shows a wide spectrum of its capabilities (though not all of it!). As an Ads Data Analyst, I will show Sroka usage in a project that leveraged advertising technology. If you are interested in how to use Sroka to access specific data source, feel free to jump straight to the respective section.

    Before diving into the code I would like to point out major benefits of using Sroka in data analyses:

    • Simplicity - after initia…

    Read more >
  • Amelable

    Optimizing the advertising strategy has recently become even more challenging than it used to be. The main reason for this is the overall declining CTR, which is the ratio between clicks and impressions, one of the key metrics in digital advertising. One cause for this observed trend might be the growing importance of the web as a sales channel and the resulting need to advertise online. The amount of unique online advertisements has grown tremendously over the years, severely impacting the user experience on websites¹. Ads became more and more intrusive due to the competition, resulting in a growing amount of people using ad-blocking tools². This trend caused the development of the banner blindness phenomenon which describes the fact that…

    Read more >
Community content is available under CC-BY-SA unless otherwise noted.

Fandom may earn an affiliate commission on sales made from links on this page.

Stream the best stories.

Fandom may earn an affiliate commission on sales made from links on this page.

Get Disney+