Posts on Random Ramblings

Using Opus 5 Without Going Insane

Fri, 18 Sep 2026 00:00:00 +0000

I’ve been using Opus 5 as my daily driver for a few weeks now across a bunch of projects and it’s a really capable model, but it has two habits that drove me a little nuts before I figured out how to deal with them:

It is verbose in a way that actually hides the answer. You ask a yes or no question and get four paragraphs of context, a code snippet, a caveat, and then the answer somewhere in the middle.
When it doesn’t understand something it does not go look it up, it guesses, and then it builds a lot of confident sounding structure on top of the guess.

This isn’t a post about whether Opus 5 is good or not (it is), it’s strictly what I do day to day to keep it useful. None of this is clever, most of it is just repeating yourself.

New Minimal Dependency vimrc with ALE

Thu, 06 Oct 2022 00:00:00 +0000

New Minimal Dependency vimrc with ALE

I stumbled onto a good simple adjustment to my minimal config while trying to figure out a good vim setup for Java, Rust, and Go. Yes I know I can just use VSCode and everything will just work, but for my home stuff I like to keep things relatively stripped down, maybe a few packages via homebrew to keep things up to date, but I no longer like to spend oodles of time every time I log in running updates on a hundred different packages, vscode often feels like this to me, a chain of updates if I haven't logged in for awhile.

Confirmed Daily Writing is Too Much

Sun, 17 Jul 2022 00:00:00 +0000

Confirmed Daily Writing is Too Much

Not much to say here, but my little experiment lasted a week (and even then I missed a day).

While I admit this is starting to pull opinions out of the back of my head and that is useful and I do see the appeal for others, but I think meditation can give me that.

I think a good middle ground maybe every weekend. I believe I have some good or at least interesting ideas to document for myself and sometimes link to others, but I was really reaching for ideas by day 5.

Why Free Software Still Matters to Me

Sat, 16 Jul 2022 00:00:00 +0000

Why Free Software Still Matters to Me

When I was younger I had very little money, this was one part bad decision and one part background that I had to overcome. I got my first computer as an adult on a loan from an aunt who had saved all her money and wanted to make sure her nephew could go back to university (I never finished sorry auntie, but I think I did plenty well due to that loan, it all worked out extremely well.)

Ukrainian Proving Easy And Hard 1st Week

Fri, 15 Jul 2022 00:00:00 +0000

Ukrainian Proving Easy And Hard 1st week

About a week in with my 2nd try and a lot of things are sticking that I thought would be impossible. I’ll touch on those points and hope I am right as I am just beginning in this language.

The easier than expected

Changing endings for nouns to say something is on or in, not super hard, seems pretty intuitive really as the sounds connect the words together.
Pronunciation is often super easy and I feel like I’ve already got a very good accent in most of what I’ve learned. At times it reminds me of European Portuguese.

The harder than expected

the combination of endings and beginnings to words changes a lot. To say “also” changes to entirely different sounds depending on what you are doing or what noun you’re working with.
Pronunciation is SOMETIMES insanely hard and a given word will have effectively tons of syllables with several “kh” “ts” “shch” combos in it at once. I sometimes forget the other words I’m trying to say while I am focusing on a massively long word.

Week 2 strategy

I plan on drilling more vocabulary but the main focus is still getting the new sounds right. This work is the most important at the beginning and if I had to do French over again it would be to focus on pronunciation for the first 6 months with little grammar, little conjugation, and just learn some travel/daily life phrases but say them very very well. I am trying this now with Ukrainian.

Coding in Lieu of Scripting in Practice

Wed, 13 Jul 2022 00:00:00 +0000

Coding in Lieu of Scripting in Practice

I wrote a short blog yesterday about how I approach the problem of munging large amount of log data without just relying on grep, sed or awk to do my troubleshooting. I would like to talk about how that tends to actually work out.

finding patterns

Take the following log from a random server that maybe has issues.


INFO [main] 2019-12-10 22:37:19,773 YamlConfigurationLoader.java:89 - Configuration location: file:/home/ubuntu/apache-cassandra-3.11.5/conf/cassandra.yaml
INFO [main] 2019-12-10 22:37:20,087 Config.java:516 - Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; back_pressure_enabled=false; back_pressure_strategy=org.apache.cassandra.net.RateBasedBackPressure{high_ratio=0.9, factor=5, flow=FAST}; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=null; broadcast_rpc_address=null; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=0; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=null; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=NaN; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@3ee0fea4; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_materialized_views=true; enable_sasi_indexes=true; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_round_up=null; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=dc; internode_recv_buff_size_in_bytes=0; internode_send_buff_size_in_bytes=0; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=localhost; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=0; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_flush_in_batches_legacy=true; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_concurrent_requests_in_bytes=-1; native_transport_max_concurrent_requests_in_bytes_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_negotiable_protocol_version=-2147483648; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=256; otc_backlog_expiration_interval_ms=200; otc_coalescing_enough_coalesced_messages=8; otc_coalescing_strategy=DISABLED; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; repair_session_max_tree_depth=18; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=localhost; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=null; seed_provider=org.apache.cassandra.locator.SimpleSeedProvider{seeds=127.0.0.1}; server_encryption_options=<REDACTED>; slow_query_log_timeout_in_ms=500; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_keep_alive_period_in_secs=300; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@48524010; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
INFO [main] 2019-12-10 22:37:20,088 DatabaseDescriptor.java:381 - DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
INFO [main] 2019-12-10 22:37:20,088 DatabaseDescriptor.java:439 - Global memtable on-heap threshold is enabled at 919MB
INFO [main] 2019-12-10 22:37:20,089 DatabaseDescriptor.java:443 - Global memtable off-heap threshold is enabled at 919MB
WARN [main] 2019-12-10 22:37:20,243 DatabaseDescriptor.java:579 - Only 54.609GiB free across all data volumes. Consider adding more capacity to your cluster or removing obsolete snapshots
INFO [main] 2019-12-10 22:37:20,281 RateBasedBackPressure.java:123 - Initialized back-pressure with high ratio: 0.9, factor: 5, flow: FAST, window size: 2000.
INFO [main] 2019-12-10 22:37:20,281 DatabaseDescriptor.java:773 - Back-pressure is disabled with strategy org.apache.cassandra.net.RateBasedBackPressure{high_ratio=0.9, factor=5, flow=FAST}.
INFO [main] 2019-12-10 22:37:20,430 JMXServerUtils.java:246 - Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
INFO [main] 2019-12-10 22:37:20,439 CassandraDaemon.java:476 - Hostname: ryan-java-devINFO [main] 2019-12-10 22:37:20,439 CassandraDaemon.java:483 - JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_222
INFO [main] 2019-12-10 22:37:20,440 CassandraDaemon.java:484 - Heap size: 3.592GiB/3.592GiB
INFO [main] 2019-12-10 22:37:20,440 CassandraDaemon.java:489 - Code Cache Non-heap memory: init = 2555904(2496K) used = 4285440(4185K) committed = 4325376(4224K) max = 251658240(245760K)INFO [main] 2019-12-10 22:37:20,441 CassandraDaemon.java:489 - Metaspace Non-heap memory: init = 0(0K) used = 17108928(16707K) committed = 17694720(17280K) max = -1(-1K)INFO [main] 2019-12-10 22:37:20,441 CassandraDaemon.java:489 - Compressed Class Space Non-heap memory: init = 0(0K) used = 2056736(2008K) committed = 2228224(2176K) max = 1073741824(1048576K)
INFO [main] 2019-12-10 22:37:20,441 CassandraDaemon.java:489 - Par Eden Space Heap memory: init = 671088640(655360K) used = 161072712(157297K) committed = 671088640(655360K) max = 671088640(655360K)INFO [main] 2019-12-10 22:37:20,442 CassandraDaemon.java:489 - Par Survivor Space Heap memory: init = 83886080(81920K) used = 0(0K) committed = 83886080(81920K) max = 83886080(81920K)INFO [main] 2019-12-10 22:37:20,442 CassandraDaemon.java:489 - CMS Old Gen Heap memory: init = 3101687808(3028992K) used = 0(0K) committed = 3101687808(3028992K) max = 3101687808(3028992K)
INFO [main] 2019-12-10 22:37:20,442 CassandraDaemon.java:491 - Classpath: /home/ubuntu/apache-cassandra-3.11.5/bin/../conf:/home/ubuntu/apache-cassandra-3.11.5/bin/../build/classes/main:/home/ubuntu/apache-cassandra-3.11.5/bin/../build/classes/thrift:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/HdrHistogram-2.1.9.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/ST4-4.0.8.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/airline-0.6.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/antlr-runtime-3.5.2.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/apache-cassandra-3.11.5.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/apache-cassandra-thrift-3.11.5.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/asm-5.0.4.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/caffeine-2.2.6.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/cassandra-driver-core-3.0.1-shaded.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/commons-cli-1.1.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/commons-codec-1.9.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/commons-lang3-3.1.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/commons-math3-3.2.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/compress-lzf-0.8.4.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/concurrent-trees-2.4.0.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/concurrentlinkedhashmap-lru-1.4.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/disruptor-3.0.1.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/ecj-4.4.2.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/guava-18.0.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/high-scale-lib-1.0.6.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/hppc-0.5.4.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/jackson-core-asl-1.9.13.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/jackson-mapper-asl-1.9.13.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/jamm-0.3.0.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/javax.inject.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/jbcrypt-0.3m.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/jcl-over-slf4j-1.7.7.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/jctools-core-1.2.1.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/jflex-1.6.0.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/jna-4.2.2.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/joda-time-2.4.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/json-simple-1.1.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/jstackjunit-0.0.1.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/libthrift-0.9.2.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/log4j-over-slf4j-1.7.7.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/logback-classic-1.1.3.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/logback-core-1.1.3.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/lz4-1.3.0.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/metrics-core-3.1.5.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/metrics-jvm-3.1.5.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/metrics-logback-3.1.5.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/netty-all-4.0.44.Final.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/ohc-core-0.4.4.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/ohc-core-j8-0.4.4.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/reporter-config-base-3.0.3.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/reporter-config3-3.0.3.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/sigar-1.6.4.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/slf4j-api-1.7.7.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/snakeyaml-1.11.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/snappy-java-1.1.1.7.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/snowball-stemmer-1.3.0.581.1.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/stream-2.5.2.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/thrift-server-0.3.7.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/jsr223/*/*.jar:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/jamm-0.3.0.jar
INFO [main] 2019-12-10 22:37:20,443 CassandraDaemon.java:493 - JVM Arguments: [-Xloggc:/home/ubuntu/apache-cassandra-3.11.5/bin/../logs/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+UseNUMA, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseParNewGC, -XX:+UseConcMarkSweepGC, -XX:+CMSParallelRemarkEnabled, -XX:SurvivorRatio=8, -XX:MaxTenuringThreshold=1, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:CMSWaitDuration=10000, -XX:+CMSParallelInitialMarkEnabled, -XX:+CMSEdenChunksRecordAlways, -XX:+CMSClassUnloadingEnabled, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Xms3757M, -Xmx3757M, -Xmn800M, -XX:+UseCondCardMark, -XX:CompileCommandFile=/home/ubuntu/apache-cassandra-3.11.5/bin/../conf/hotspot_compiler, -javaagent:/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/home/ubuntu/apache-cassandra-3.11.5/bin/../lib/sigar-bin, -XX:OnOutOfMemoryError=kill -9 %p, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/home/ubuntu/apache-cassandra-3.11.5/bin/../logs, -Dcassandra.storagedir=/home/ubuntu/apache-cassandra-3.11.5/bin/../data]
INFO [main] 2019-12-10 22:37:20,939 NativeLibrary.java:176 - JNA mlockall successful
WARN [main] 2019-12-10 22:37:20,940 StartupChecks.java:136 - jemalloc shared library could not be preloaded to speed up memory allocations
WARN [main] 2019-12-10 22:37:20,941 StartupChecks.java:169 - JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
INFO [main] 2019-12-10 22:37:20,944 SigarLibrary.java:44 - Initializing SIGAR library
INFO [main] 2019-12-10 22:37:20,959 SigarLibrary.java:180 - Checked OS settings and found them configured for optimal performance.
WARN [main] 2019-12-10 22:37:20,963 StartupChecks.java:311 - Maximum number of memory map areas per process (vm.max_map_count) 65530 is too low, recommended value: 1048575, you can change it with sysctl.
WARN [main] 2019-12-10 22:37:20,974 StartupChecks.java:332 - Directory /home/ubuntu/apache-cassandra-3.11.5/bin/../data/data doesn't exist
WARN [main] 2019-12-10 22:37:20,981 StartupChecks.java:332 - Directory /home/ubuntu/apache-cassandra-3.11.5/bin/../data/commitlog doesn't exist
WARN [main] 2019-12-10 22:37:20,982 StartupChecks.java:332 - Directory /home/ubuntu/apache-cassandra-3.11.5/bin/../data/saved_caches doesn't exist
WARN [main] 2019-12-10 22:37:20,983 StartupChecks.java:332 - Directory /home/ubuntu/apache-cassandra-3.11.5/bin/../data/hints doesn't exist

This is a Cassandra startup log I have taken from an old computer. This is only the first few lines and not even close to a full startup log. Most would agree, it is still a great deal of text.

Coding in Lieu of Scripting

Tue, 12 Jul 2022 00:00:00 +0000

Coding in Lieu of Scripting

Since I "mostly" moved to support about 8 years ago now I have parsed a LOT of logs, and I enjoy it. I like to search for unknown problems before they become known at 3am, or figure out what happened exactly at a given point in time, especially when no one else seems to know. The discovery is fun and I get to tell a story with the data. At first I used the shell at lot and I rediscovered lost skills from my early days in ISP land. However, I found it very difficult to share that knowledge. Scripts were never quite portable enough, and the Windows users would just stare at me in bewilderment when I handed them a bash script Note this is before WSL was really a thing, and while I enjoy powershell support is unix centric.

Slavic Languages

Mon, 11 Jul 2022 00:00:00 +0000

Slavic Languages

I have just started toying with Slavic Languages. Due to my family tree this has always been a bit of a toy interest of mine, butI never pursued it seriously as there seemed little need. The war in Ukraine has changed that focus and from that point on I’ve had at least one Slavic language going in the background of my language learning.

Goals

I would like to be able to follow the events in Ukraine, Poland and Russia more directly and without translation.
Be able to have a brief conversation with anyone in those three languages someday. Mastery is unlikely while living in a country where I need to survive in a different non native language to my target language (so my normal strategy of full immersion is out).
If need be, help out with refugees here in France, this seems somewhat unnecessary now as of yet there are enough native speakers around to satisfy needs as the expected massive influx of Ukrainians never fully materialized in France for a variety of reasons. However, the future is hard to see, may as well get started.

Initial thoughts

I am not even the slightest of an expert in any of this and I’ve only studied some linguistics aimed at the general public. I do have a pretty considerable exposure to Latin languages now and I’m not a newbie to language learning, so I think I may have something valuable to add.

fallen into rust

Sun, 10 Jul 2022 00:00:00 +0000

fallen into rust

I have just fallen into my first days of the Rust programming language. It wasn’t as difficult as I feared and I wrote a high performance log reporter and analyzer. It took longer than normal to write and I did waste 2 days trying to understand how variable life times worked and this “borrow checker” thing but I managed to write a fast and seemingly major bug free prototype that is getting some heavy usage.

Post a day

Sat, 09 Jul 2022 00:00:00 +0000

Post a day

I have read that people try to write daily, and I always thought this was a bit silly, even if laudable. The idea of trying to take something and make it a habit is commendable. Still, I am not sure anyone will have high-quality writing every day. Furthermore, my state of mind changes so often it is difficult for me to do anything once a day. It is just not natural for me, even if I have known plenty of people who seemed to thrive on daily habits for decades.

Language Learning Strategy 2022

Thu, 26 May 2022 00:00:00 +0000

Language Learning Strategy 2022

Taking a big departure from my traditional blogging to talk about something probably a bit less interesting for those of you who still read this for tech, sorry, you can just skip this one if you want to read up on tech.

Where I am at

Today I have pretty middling levels in three languages French, Spanish and Portuguese (I can have conversations in all of them, with errors of course). A common question is why don't you focus on one and master it? Fair question, Spanish I got pretty close to advanced Spanish, but life got in the way and so picked up Portuguese, that got pretty good then again life changed plans and so then came French out of necessity.

Trying to Get Away From Jekyll

Wed, 30 Mar 2022 00:00:00 +0000

Trying to Get Away From Jekyll

This morning I tried to switch to pure HTML from Jekyll on GitHub pages and it was not trivial, I finally punted and gave up but I wanted to share the brick wall I ran into so I can try again later.

why?

Access

Simple, I am trying to use my computer less. When I have a thought I would like to share and say I have my phone on me, I would just like to rip out a blog post, without setting up git, getting a decent text editor, etc it's just too much work to do well.

Getting Index Size in DSE Search

Thu, 24 Mar 2022 02:00:00 +0000

Getting Index Size in DSE Search

Applies to DSE Search versions 3.2 to 6.8

For some reason finding this information is dreadfully stupidly hard and I finally decided to write it down myself since I own it and someone may benefit from it some day. There are three ways that I know of (I am sure there is a JMX way I just do not happen to know it atm)

Busy Engineers Guide to Deploying on K8s Part 1 - Setting up Docker with GitHub Packages

Tue, 12 Oct 2021 00:00:00 +0000

Busy Engineers Guide to Deploying on K8s Part 1 - Setting up Docker with GitHub Packages

Post Valid as of October 12, 2021.

Busy Engineer’s Guide to Deploying On K8s

While trying to take some coworkers through setting up Docker with GitHub Packages I ran some not obvious edges and my boss pointed out for new to Docker and new to Kubernetes people these could be totally derailing corners. I have to agree, so this series will take people front to back through setting up Kubernetes with the options most readily available to us.

Busy Engineers Guide to Deploying on K8s Part 2 - Kind

Tue, 12 Oct 2021 01:00:00 +0000

Busy Engineers Guide to Deploying on K8s Part 2 - Kind

Post Valid as of October 12, 2021.

Busy Engineer’s Guide to Deploying On K8s

This is a continuation from the last post in the series. There are three primary ways to run Kubernetes locally on a notebook: Kind, k3d, and Minikube. I personally far prefer k3d due to it having good defaults to get up and running with stateful sets long before the other two and it can create large worker nodes, these have lead to a loyalty to the product. However, my org seems to have standardized on Kind for most of their CI so I will be covering that here as it is the easiest for my coworkers to find help and documentation. It is also an excellent product to get started with Kubernetes.

Busy Engineers Guide to Deploying on K8s Part 3 - Helm

Tue, 12 Oct 2021 02:00:00 +0000

Busy Engineers Guide to Deploying on K8s Part 3 - Helm

Post Valid as of October 12, 2021.

Busy Engineer’s Guide to Deploying On K8s

Ok so now that you have Kubernetes installing in a deployment and you can see it with kubectl port-forward how to you get beyond the toy stage?

Helm Charts

Helm is a great way to make your app “production ready” on Kubernetes with minimal effort assuming you do not try to overthink your deployment. Helm fundamentally provides a way for us to deploy apps to kubernetes by easily changing a handful of values at deployment time or in something called a values.yaml

Detecting and dealing with backpressure in DSE 6.x

Wed, 28 Jul 2021 08:11:00 +0000

Detecting and dealing with backpressure in DSE 6.x

Several years ago DataStax change DataStax Enterprise from a version of Cassandra with some closed source plugins and a handful of fixes most of which were backports of fixes in very old versions, to a fundamentally different architecture. This new architecure we will refer to as TPC, however, there were also several internals that were changed that may have relied or depended on TPC but were not actually themselves part of TPC.

Creating a Spark cluster with Cass Operator

Tue, 08 Jun 2021 00:00:00 +0000

Creating a Spark cluster with Cass Operator

This is not officially supported as of June 8, 2021 but I am including it anyway since it is not obvious and documented anywhere. There are cases where for some POC or simple thing you want to test out in development this maybe a desirable thing to have handy (I use it to run clusters on my notebook).

First install the cass-operator by running the following helm commands

Random Kubernetes Failures and Node Size

Tue, 09 Mar 2021 00:00:00 +0000

Random Kubernetes Failures and Node Size

I had a report of ingress failing on traefik randomly during a test job and since it worked on a local workstation the belief was it had to be a problem with traefik. I looked at the logs and asked them to spin up bigger nodes for the test and it fixed things.

The reason the beefier machines come into play are varied but ultimately come down to “no free lunch”. It all started with these log entries in traefik:

The Joy and Pain of Swagger

Fri, 05 Mar 2021 00:00:00 +0000

The Joy and Pain of Swagger

I have had to use Swagger for many a project since my first introduction to it while working for an ecommerce site in 2015. I did like being able to easily convert documentation into viewable API docs complete with executable examples and forms ready to go that let me actually try out my new API. In many ways it is the closest to the promise of API first development.

Why I Don't Cobra

Thu, 04 Mar 2021 00:00:00 +0000

Why I Don't Cobra

So I just implemented this DataStax Astra CLI interface and it has gotten some attention where I work. One of the developers who had worked on a similar tool had suggested I use cobra to build a command line interface.

Code size

I explained to him that I actually had a lot of experience with Cobra and that I thought it was an excellent library with lots of great support for nested commands, it also had lots of transitive dependencies (313 lines in the go.sum at the time of this post), and I’ve discovered that it actually can bloat the size of the executable quite a bit. Therefore I had elected to use the standard library to build my command line flags. Admittedly, I’ve had to do a lot more work to actually build my commandline interface, I did kept the executable size to under 4 mb for all platforms.

Tuning Indexing Speed in DSE Search 5.x

Fri, 26 Feb 2021 00:00:00 +0000

Tuning Indexing Speed in DSE Search 5.x

What follows is a summary of the things you need to consider dealing with indexing speed problems in DSE 5.x (note the model of indexing is wildly different in DSE 6.x and will not apply, I will have a guide on that soon).

Dropped Mutations

With Solr 5.x dropped mutations have three causes over regular Cassandra:

During the merging of Solr indexes, you can “stall” due to configuration limits.
By default, Solr indexing is very aggressive and can starve IO, all activities will slow, and the cluster may experience dropped mutations.
Back pressure leading to drops. If QueueWeight in the com.datastax.bdp.search.keyspace_name.table_name.IndexPool MBean is greater than back_pressure_threshold_per_core, back pressure is happening.

To minimize IO contention and merge stalling do the following steps:

A Tale of Two Vims

Tue, 23 Feb 2021 00:00:00 +0000

A Tale of Two Vims

I have two primary vim onfigurations depending on what I am feeling. I have my “everything is modern” configuration that I will bring out if I need to get a lot of work done. Which is below. This has autocomplete, nice dark theme, git integration, airline like, nerdtree, ctrlp, and with COC it has strong support for nearly every language. It is like running VSCode in the terminal, works fantastique.

Configuring UFW with Mosh support

Tue, 23 Feb 2021 00:00:00 +0000

Configuring UFW with Mosh support

Mosh is kind of my secret weapon for working remotely but getting it configured to be allowed through the firewall is something I always seem to forget. So here is my UFW script I use for new servers with mosh added in:

sudo ufw default deny incoming
sudo ufw allow 22/tcp
# this is hte part that allows mosh to work
sudo ufw allow 60000:61000/udp
sudo ufw enable

Getting the DataStax cass-operator working on Mac

Tue, 23 Feb 2021 00:00:00 +0000

Getting the DataStax cass-operator working on Mac

Just leaving this here for anyone coming back to this later. This works as of Feb 23 2021 on Big Sur

install docker desktop https://www.docker.com/products/docker-desktop
install k3d https://github.com/rancher/k3d#get you can just run curl -s https://raw.githubusercontent.com/rancher/k3d/main/install.sh | bash
create cluster: k3d cluster create
install operator manifest kubectl create -f https://raw.githubusercontent.com/datastax/cass-operator/v1.5.1/docs/user/cass-operator-manifests-v1.19.yaml
download a cassandra data center yaml curl -O https://gist.githubusercontent.com/rsds143/0e9263ed4287d888fab36eb3e4aec502/raw/f919172c64452a2277b7523d680a9e12916d0d39/cassandra-dc.yaml
run kubectl create -f cassandra-dc.yaml --namespace cass-operator
wait for cassandra to come up kubectl logs cluster1-dc1-default-sts-0 --namespace cass-operator server-system-logger
get username export CQLUSER=$(kubectl get secret cluster1-superuser -o json -n cass-operator | grep \"username\" | sed -E 's/\"(.+)\": \"(.+)\"/\2/' | base64 -d)
get password export CQLPASS=$(kubectl get secret cluster1-superuser -o json -n cass-operator | grep \"password\" | sed -E 's/\"(.+)\": \"(.+)\"/\2/' | sed -E 's/,*$//g' | base64 -d )
log into cqlsh kubectl exec -it -n cass-operator cluster1-dc1-default-sts-0 -- cqlsh -u $CQLUSER -p $CQLPASS
enjoy your cassandra cluster

The yaml I have linked too above is here

minimal zsh prompt that is useful

Tue, 23 Feb 2021 00:00:00 +0000

minimal zsh prompt that is useful

So oh-my-zsh is great but I like to do as much as I can on my own So I came up with a basic zsh prompt that gives me enough information to know what server I am on and what directory I am in (also git branch)

Enjoy


autoload -Uz vcs_info
precmd() { vcs_info }

# Format the vcs_info_msg_0_ variable
zstyle ':vcs_info:git:*' formats ' %b'
setopt PROMPT_SUBST
# export PATH=$HOME/bin:/usr/local/bin:$PATH
# 
# %(?.√.?%?) if return code ? is 0, show √, else show ?%?
# %? exit code of previous command
# %1~ current working dir, shortening home to ~, show only last 1 element
# %# # with root privileges, % otherwise
# %B %b start/stop bold
# %F{...} text (foreground) color, see table
# %f reset to default textcolor
export PROMPT='%(?.%F{green}√.%F{red}?%?)%f %B%F{240}%1~%f%b@%F{184}%m%f %F{green}${vcs_info_msg_0_}%f %# '

export PATH=$PATH:$HOME/go/bin:$HOME/.cargo/bin

Quick IP scan without nmap

Tue, 23 Feb 2021 00:00:00 +0000

Quick IP scan without nmap

So nmap isn’t always available to figure out what devices are up on your local network or where they may have moved too after you installed that new router but python is usually around. This script will ping your local network for ip addresses. It is as basic as you can get and does not work on Windows. Some day I would like to make a better version of this, when I do, I will link it back to here. In the meantime this maybe useful.

Apache NiFi and Cassandra

Mon, 08 Feb 2021 00:00:00 +0000

Apache NiFi and Cassandra

Looked at Apache NiFi for the first time in a long time as someone didn’t know if it had support for stored procedures or not. I looked up the latest source and sure enough I see no stored procedures used for any of the statements. Looking at the PutCassandraRecord.java I see inserts and updates are just statements that are never prepared:

 query = generateInsert(cassandraTable, schema, recordContentMap);

and

Heap Size Around 32 GB

Wed, 03 Feb 2021 00:00:00 +0000

Heap Size Around 32 GB

note: originally written Feb 3 2021

There was a question this morning around a pretty common discussion point around 31 gb versus 32 gb of heap. This article discusses this in more detail, the key point follows:

Be careful when you increase your application heap size over 32G (from under 32G to over 32G) – JVM switches to 64 bit object references at that moment, which means that your application may end up with less available heap space. A rule of thumb is to jump from 32G right to 37-38G and continue adding memory from that point. The actual area of “grey” territory depends on your application – the bigger an average Java object in your application, the smaller is the overhead.

coc-vim the perfect compliment to vim

Sun, 14 Jun 2020 00:00:00 +0000

coc-vim the perfect compliment to vim

Vi(m) has been the reliable editor of my career. I’ve either used Vi bindings in my IDEs or used Vim itself for my editor of choice. Some of this was because I was working on servers a lot during my career as an admin, sometimes it was because the dynamic languages I liked using don’t or didn’t have great IDE support so may as well use something I’m familar with and is light weight.

Picking a web microframework for an existing Scala codebase

Wed, 27 May 2020 00:00:00 +0000

Picking a web microframework for an existing Scala codebase

I’ve had to use this at work the last couple of weeks. We had a “home grown” framework for a new application we’re working and the first thing I did was try and rip that out (new project so didn’t have URL and parameter sanitization anyway to do routes, etc).

However, being that the group I was working had bad expierences in the past with frameworks, I had to settle on something that was light weight, integrated with jetty and allowed us to work the way that was comfortable for us as team (also it had to work with Scala).

Starting Cassandra - Data modeling in the brief

Wed, 05 Feb 2020 00:00:00 +0000

Starting Cassandra - Data modeling in the brief

Cassandra data modeling isn’t really something you can do “in the brief” and is itself a subject that can take years to fully grasp, but this should be a good starting point.

Introduction

Cassandra distributes data around the cluster via the partition key.

CREATE TABLE my_key.my_table_by_postal_code (postal_code text, id uuid, balance float, PRIMARY KEY(postal_code, id));

In the above table the partition key is postal_code and the clustering column isid. The partition key will locate the data on the cluster for us. The clustering column allows us multiple rows per partition key so that we can filter how much data we read per partition. The ‘optimal’ query is one that retrieves data from only one node and not so much data that GC pressure or latency issues result. The following query is breaking that rule and retrieving 2 partitions at once via the IN parameter.

Starting Cassandra - Load testing Cassandra in brief

Tue, 04 Feb 2020 00:00:00 +0000

Starting Cassandra - Load testing Cassandra in brief

An opinionated guide on the “correct” way to load test Cassandra. I’m aiming to keep this short so I’m going to leave out a lot of the nuance that one would normally get into when talking about load testing cassandra.

If you have no data model in mind

Use cassandra stress since it’s around:

first initialize the keyspace with RF3 cassandra-stress "write cl=ONE no-warmup -col size=FIXED(15000) -schema replication(strategy=SimpleStrategy,factor=3)"
second run stress cassandra-stress "mixed n=1000k cl=ONE -col size=FIXED(15000)
repeat as often as you’d like with as many clients as you want.

If you have a specific data model in mind

You can use cassandra-stress, but I suspect you’re going to find your data model isn’t supported (collections for example) or that you don’t have the required PHD to make it work the way you want. There are probably 2 dozen options from here you can use to build your load test, some of the more popular ones are gatling, jmeter, and tlp-stress. My personal favorite for this though, write a small simple python or java program that replicates your use case accurately in your own code, using a faker library to generate your data. This takes more time but you tend to have less surprises in production as it will accurately model your code.

Starting Cassandra - Setting up a Multi-DC environment

Mon, 03 Feb 2020 00:00:00 +0000

Starting Cassandra - Setting up a Multi-DC environment

This is a quick and dirty opinionated guide to setting up a Cassandra cluster with multiple data centers.

A new cluster

In cassandra.yaml set endpoint_snitch: GossipingPropertyFileSnitch, some prefer PropertyFileSnitch for the ease of pushing out one file. GossipingPropertyFileSnitch is harder to get wrong in my experience.
set dc in cassandra-rackdc.properties. Set to be whatever dc you want that node to be in. Ignore rack until you really need it, 8/10 people that use racks do it wrong the first time, and it’s slightly painful to unwind.
finish adding all of your nodes.
if using authentication, set system_auth keyspace to use NetworkTopologyStrategy in cqlsh with RF 3 (or == number of replicas if less than 3 per dc) for each datacenter you’ve created ALTER KEYSPACE system_auth WITH REPLICATION= {'class' : 'NetworkTopologyStrategy', 'data_center_name' : 3, 'data_center_name' : 3};, run repair after changing RF
nodetool repair -pr system_auth on each node in the cluster on the new keyspace.
create your new keyspaces for your app with RF 3 in each dc (much like you did for the system_auth step above).
nodetool repair -pr whatever_new_keyspace on each node in the cluster on the new keyspace.

An existing cluster

This is harder and involves more work and more options, but I’m going to discuss the way that gets you into the least amount of trouble operationally.

Snakeyaml error handling is tough

Fri, 27 Dec 2019 09:30:00 -0600

"Snakeyaml error handling is tough"

Cassandra for years has relied on snakeyaml to parse configuration files, and it does a fine job, really not a lot of complaints. Until one day a colleague of mine discovered an issue where a permission issue incorrectly reported as an invalid yaml file (complete with the supposedly offending key). Digging into the source code and trying to match up the message. I found that we were handling YAMLException

MVP how minimal

Thu, 20 Dec 2018 20:00:00 +0000

MVP how minimal

MVPs or Minimum Viable Products are pretty contentious ideas for something seemingly simple. Depending on background and where pepole are coming from experience wise those terms carry radically different ideas. In recent history I've seen up close two extreme constrasting examples of MVP:

Mega Minimal: website and db, mostly manual on the backend
Mega Mega: provisioning system, dynamic tuning of systems via ML, automated operations, monitoring a few others I’m leaving out.

Feedback

If we’re evaluating which approach gives us more feedback, Mega Minimal MVP is gonna win hands down here. Some will counter they don’t want to give people a bad impression with a limited product and that’s fair, but it’s better than no impression (the dreaded never shipped MVP). The Mega Mega MVP I referenced took months to demo. only had one of those checkboxes setup and wasn’t ever demod again. So we can categorical say that failed at getting any feedback.

Surprise Go is ok for me now

Thu, 13 Dec 2018 20:23:00 +0000

Surprise Go is ok for me now

EDIT: this is an old post from my old blog I have updated it here and ported it over here for archival reasons.

I’m surprised to say this. I am ok using Go now. It’s not my style, but I can build almost anything I want to with it, and the tooling around it continues to improve.

About seven months ago, I wrote about all the things I didn’t care for in Go and now I either no longer am so bothered by it, or things have improved.

Lessons from a year of Golang

Mon, 07 May 2018 13:16:00 +0000

Lessons from a year of Golang

I’m hoping to share in a non-negative way help others avoid the pitfalls I ran into with my most recent work building infrastructure software on top of a Kubernetes using Go, it sounded like an awesome job at first but I ran into a lot of problems getting productive.

This isn’t meant to evaluate if you should pick up Go or tell you what you should think of it, this is strictly meant to help people out that are new to the language but experienced in Java, Python, Ruby, C#, etc and have read some basic Go getting started guide.

Cassandra - Batch Loading Without the Batch The Nuanced Edition

Fri, 29 Apr 2016 16:03:56 +0000

Cassandra - Batch Loading Without the Batch The Nuanced Edition

My [previous post on this subject](https://blog.foundev.pro/2014/08/28/cassandra-batch-loading-without-the-batch-keyword.html) has proven extraordinarily popular and I get commentary on it all the time, most of it quite good. It has however, gotten a decent number of comments from people quibbling with the nuance of the post and pointing out it’s failings, which is fair because I didn’t explicitly spell this out as a “framework of thinking” blog post or a series of principles to consider. This is the tension between making something approachable and understandable to the new user but still technically correct for the advanced one. Because of where Cassandra was at the time and the user base I was encountering day to day, I took the approach of simplification for the sake of understanding. However, now I think is a good time to write this up with all the complexity and detail of a production implementation and the tradeoffs to consider.

TLDR

Find the ideal write size it can make a 10x difference in perf (10k-100k is common).
Limit threads in flight when writing.
Use tokenaware unlogged batches if you need to get to your ideal size.

Details on all this below.

CASSANDRA LOCAL_QUORUM SHOULD STAY LOCAL

Thu, 21 Apr 2016 00:00:00 +0000

CASSANDRA LOCAL_QUORUM SHOULD STAY LOCAL

A couple of times a week I get a question where someone wants to know how to “failover” to a remote DC in the driver if the local Cassandra DC fails or even if there is only a couple of nodes in the local data center that are down.

Frequently the customer is using LOCAL_QUORUM or LOCAL_ONE at our suggestion to pin all reads and writes to the local data center often times at their request to help with p99 latencies. But now they want the driver to “do the right thing” and start using replicas in another data center when there is a local outage.

Connection to Oracle From Spark

Sun, 10 Apr 2016 00:00:00 +0000

Connection to Oracle From Spark

For some silly reason there is a has been a fair amount of difficulty in reading and writing to Oracle from Spark when using DataFrames.

SPARK-10648 — Spark-SQL JDBC fails to set a default precision and scale when they are not defined in an oracle schema.

This issue manifests itself when you have numbers in your schema with no precision or scale and attempt to read. They added the following dialect here

Reflection Scala-2.10 and Spark weird errors when saving to Cassandra

Sat, 09 Apr 2016 00:00:00 +0000

Reflection Scala-2.10 and Spark weird errors when saving to Cassandra

This originally started with this SO question, and I’ll be honest I was flummoxed for a couple of days looking at this (in no small part because the code was doing a lot). But at some point I was able to isolate all issues down to dataFrame.saveToCassandra. Every 5 or so runs I’d get one or two errors:

I could write this data to a file with no issue, but when writing to a Cassandra these errors would come flying out at me. With the help of Russ Spitzer (by help I mean explaining to my thick skull what was going on) I was pointed to the fact that reflection wasn’t thread safe in Scala 2.10.

Logging The Generated CQL from the Spark Cassandra Connector

Thu, 07 Apr 2016 00:00:00 +0000

Logging The Generated CQL from the Spark Cassandra Connector

This has come up some in the last few days so I thought I’d share the available options and the tradeoffs.

Option 1: Turn ON ALL THE TRACING! nodetool settraceprobability 1.0

Probabilistic tracing is a handy feature for finding expensive queries in use cases where there is little control over who has access to the cluster IE most enterprises in my experience (ironic considering all the process). However, it’s too expensive to turn it up too high in production, but in development it’s a good way to give you an idea of what a query turns into. Read more about probalistic tracing here:

Don’t use TextField for your unique key in Solr

Wed, 06 Apr 2016 00:00:00 +0000

Don’t use TextField for your unique key in Solr

This seems immediately obvious when you think about it, but TextField is what you use for fuzzy searches in Solr, and why would a person want a fuzzy search on a unique value? While I can come up with some oddball use cases, making use of copy fields would seem to be the more valid approach and fitting with the typical use of Solr IE you filter on strings and query on text.

Spark job that writes to Cassandra just hangs when one node goes down?

Tue, 05 Apr 2016 00:00:00 +0000

Spark job that writes to Cassandra just hangs when one node goes down?

### So this was hyper obvious once I saw the executor logs and the database schema, but this had me befuddled at first and the change in behavior with one node should have made it obvious. {#23da}

The code was simple, read a bunch of information, do some minor transformations and flush to Cassandra. This was nothing crazy. But during the users fault tolerance testing, the job would just seamingly hang indefinitely when a node was down.

Synthetic Sharding with Cassandra. Or How To Deal With Large Partitions.

Tue, 15 Mar 2016 00:00:00 +0000

Synthetic Sharding with Cassandra. Or How To Deal With Large Partitions.

Extremely overdue that I write this down as it’s a common problem, and really applies to any database that needs to scale horizontally, not just Cassandra.

Problem Statement

Good partition keys are not always obvious, and it’s easy to create a bad one.

Defining A Bad Partition

Uneven Access. Read OR write count is more than 2x different from one partition to another. This is a purist view and some of you using time series are screaming at me right now, but set that aside, I’ll have another blog post for you, but if you’re new to Cassandra consider it a good principle and goal to aim for.
Uneven Size. Same as above really you can run cfhistograms and if you see really tiny or empty partitions next to really large ones or ones with really high cell count, you are at least on ingest uneven. I would shoot for within an order of magnitude or two. If you’re smart enough to tell me why I’m wrong here that’s fine, I’m not gonna care, but if you’re new to Cassandra this is a good goal.
Too Many Cells. The partition cell count for a given partition is over 100k (run cfhistograms to get these numbers). This is entirely a rule of thumb and varies amazingly between hardware, tuning and column names (length matters). You may find you can add more and not hit a problem and you may find you can’t get near this. If you want to be exacting you should test.
Too Large. Your partition size is over 32mb (also in cfhisograms EDIT in 2022, this is usually much larger on modern hardware). This also varies like cell count. Some people tell me this matters less now (as of Apache Cassandra™ 2.1), and they run a lot larger. However, I’ve seen it cause problems on a number of clusters. I repeat as a new user this is a good number to shoot for, once you’re advanced enough to tell me why I’m wrong you may ignore this rule. Again you should test your cluster to get the number where things get problematic.

Options if you have a bad partition

Pick a better partition key read.
Give up and use Synthetic Sharding.
Pretend it’s not a problem and find out the hard way that it really is, usually this is at 3 am.

Synthetic Sharding Strategy: Shard Table

Pros

Cassandra’s “Repair” Should Be Called “Required Maintenance

Fri, 25 Sep 2015 00:00:00 +0000

Cassandra’s “Repair” Should Be Called “Required Maintenance”

One of the bigger challenges when you go Eventually Consistent is how to reconcile data not being replicated. This happens if your using Oracle and multi-data centers with tech like Golden Gate and it happens if you’re using async replicas with MySQL and one of your replicas got out of whack. You need a way to “repair” the lost data.

Since Cassandra fully embraces eventually consistency repair is actually a important mechanism for making sure copies of data are shipped around the cluster to meet your specified replication factor. About now several of you have alarm bells going off and think I’m insane. Let’s step through the common objections I hear.

Scale is a Dish Best Served Eventually Consistent

Wed, 23 Sep 2015 00:00:00 +0000

Scale is a Dish Best Served Eventually Consistent

A lot of people new to Cassandra find the data modeling required tedious and outrageously hard. They’ll long for their RDMBS, if only insert favorite vendor or project lead here would make their RDBMS scale like Cassandra they could tell their bosses to shove off and go back to using their favorite tool. To people saying this (and I hear the echos of it in conversations often), I’ve got news for you, the problem isn’t Cassandra or your favorite tool, it’s your style of data modeling.

Data Density! Destroyer of Scalability

Thu, 17 Sep 2015 00:00:00 +0000

Data Density! Destroyer of Scalability

UPDATE: I’d incorrectly attributed a practice to Netflix about scaling down daily . I cannot find any reference to using today, and I’ve been unable to find the previous reference to it. So I’ve just removed the point. I’ll cover cluster scaling strategies later in another blog post. TLDR smaller nodes are easier to scale.

My past couple of years I’ve seen a growing and to my mind unfortunate trend towards ever larger Cassandra nodes running more expensive and more specialized hardware. Teams wanting 17TB nodes in a quest to get ever more data on a single unit (TLDR that rarely works). While wanting to lower one’s rack footprint sounds hyper appealing let me be the first to tell you, it comes at a great cost.

DataStax a Love Letter

Thu, 17 Sep 2015 00:00:00 +0000

DataStax a Love Letter

Today is my last day at DataStax and what an amazing ride it was. This is easily the best job and the best group of people I’ve ever worked with and I’m very sad to go, but I had an opportunity I could not pass up.

I want to specifically speak to some (but not all cause this would read like Numbers in the Bible) of the amazing people I’m truly blessed to know.

Event Sourcing and System of Record: Sane Distributed Development In The Modern Era

Thu, 17 Sep 2015 00:00:00 +0000

Event Sourcing and System of Record: Sane Distributed Development In The Modern Era

No matter the message queue or broker you rely on whether it be RabbitMQ, JMS, ActiveMQ, Websphere, MSMQ and yes even Kafka you can lose messages in any of the following ways:

A downstream system from the broker can have data loss
All message queues today can lose already acknowledged messages during failover or leader election.
A bug can send the wrong messages to the wrong systems.

AKA the typical distributed systems problem set. Despite these data safety issues, messaging remains a popular pattern for the following reasons:

Lambda+ or Event Sourcing with TTLs

Thu, 17 Sep 2015 00:00:00 +0000

Lambda+ or Event Sourcing with TTLs

Some people are worried about the data volume that a strategy like Lambda+ or Event Sourcing implies. As a disclaimer, by giving up the historical data you have, you risk losing useful data layer and you lose the ability to have bi-temporal data or go back in time. However, if you still don’t care about that, please continue reading.

Transaction Log Lessons

If we think of Event Sourcing as a transaction log we can borrow some valuable lessons. How is it like a transaction log you ask?

Lambda+: Cassandra and Spark for Scalable Architecture

Thu, 17 Sep 2015 00:00:00 +0000

Lambda+: Cassandra and Spark for Scalable Architecture

UPDATE: For some background on Spark Streaming and Cassandra please consult some of my previous blog post on the subject.

Many of you have heard of and a few of you may have used the Lambda Architecture. If you’ve not heard of it the name doesn’t communicate a lot. In short the Lambda Architecture has 3 primary components with a 2 pronged approach to analytics:

Real Time Analytics With Spark Streaming and Cassandra

Thu, 17 Sep 2015 00:00:00 +0000

Real Time Analytics With Spark Streaming and Cassandra

Spark Streaming is a good tool to roll up transactions data into summaries as they enter the system. When paired with an easily idempotent data store like Cassandra you get a high performance low hassle approach to getting your work done.

Useful Background

Retry Not Rollback: Idempotent Data Models in Cassandra

Event Sourcing and System Of Record (addresses Message Replay and System Of Record)

Cassandra CQL Docs on Counters

WAL: Write Ahead Logging

Direct API in Spark Streaming

General Best Practice

The safe choice will be an Idempotent Data Model that you can safely retry,Aggregate with Spark Streaming and use Message Replay driven by a System Of Record to handle any lost messages.

Retry not Rollback: Idempotent Data Models in Cassandra

Thu, 17 Sep 2015 00:00:00 +0000

Retry not Rollback: Idempotent Data Models in Cassandra

### {#737a}

Naive Consistency

Often the first error handling code I see from new Cassandra users is the client side rollback in an attempt to replicate database transactions from the ACID world. This is typically done when a write to multiple tables fail, or a write is not able to meet requested Consistency Level. Unfortunately, client side rollbacks, while well intentioned, generally create more problems than they solve. Let’s discuss how this will end up

My Cassandra 2.0 Diagnostics Checklist (Brain Dump)

Tue, 25 Nov 2014 00:00:00 +0000

My Cassandra 2.0 Diagnostics Checklist (Brain Dump)

UPDATE:

This list needs to be updated and as of today only has been verified with Cassandra 2.0.

Original Blog Post:

This isn’t remotely complete, but I had a colleague ask me to do a brain dump of my process and this is by and large usually it. I’m sure this will leave more questions than answers for many of you, and I’d like to follow up this post at some point with detail of the why and the how of a lot of it so it can be more useful to beginners. Today this is in a very raw form.

Domain Modeling Around Deletes or “Using Cassandra as a queue even when you know better

Mon, 20 Oct 2014 00:00:00 +0000

Domain Modeling Around Deletes or “Using Cassandra as a queue even when you know better”

# Understanding Deletes

Delete heavy workloads have a number of pretty serious issues when it comes to using a distributed database. Unfortunately one of the most common delete heavy workloads and the most common desired use case for Cassandra is to use it as a global queue. This blog post is aimed at addressing the queue anti-pattern primarily, but could be modeled for other delete heavy use cases.

Cassandra Auth: Never use the cassandra user in production!

Mon, 22 Sep 2014 00:00:00 +0000

Cassandra Auth: Never use the cassandra user in production!

Normal best practice for security with applications is never to use the default admin user. In Sql Server this is manifested by the recommendation not to use the “sa” user. Likewise in Cassandra the default Cassandra user has full rights to all tables and operations. This is needless to say bad security from an application design perspective, but there is yet another consideration that I see people in larger organizations run into, performance & availability.

QUORUM versus LOCAL_ONE

The default cassandra username is a special case for authentication, since it’s used to modify initial user permissions uses QUORUM consistency when logging in. In a production use case this has important performance implication for multiple data center scenarios (hereafter referered to as MDC) which means if you have 3 data centers, all with system_auth set to a replication factor of 3 (or 9 replicas total), QUORUM consistency will query 6 nodes, some of which going over high latency network links! Worse, if you were to lose 1 data center and 1 other node, you’d be unable to login!

Cassandra Query Patterns: Not using the “in” query for multiple partitions.

Mon, 22 Sep 2014 22:12:58 +0000

Cassandra Query Patterns: Not using the “in” query for multiple partitions.

So lets say you’re doing you’re best to data model all around one partition. You’ve done your homework and all you queries look like this:

 SELECT * FROM my_keyspace.users where id = 1

Over time as features are added however, you make some tradeoffs and need to start doing queries across partitions. At first there are only a few queries like this.

Apache Cassandra: Some useful JMX metrics to monitor

Wed, 03 Sep 2014 00:00:00 +0000

Apache Cassandra: Some useful JMX metrics to monitor

This is not a complete list, but is this what I’ve typically had to look out for in the wild. There maybe some selection bias at play since once I’m involved with a cluster it’s not in a good place.

_{org.apache.cassandra.metrics/DroppedMessage/MUTATION/Attributes/Count}

This should always be 0. This is indicative of load shedding and while there can be short term bursts where this is only a temporary problem that normal maintenance cleans up, it’s more often than not indicative that your cluster is overloaded, mistuned or just flat our running on bad hardware to begin with.

Cassandra C# Driver: Surprising gotcha with SimpleStatement

Fri, 29 Aug 2014 00:00:00 +0000

Cassandra C# Driver: Surprising gotcha with SimpleStatement

When helping someone with a Batch using the C# driver and I had a bit of a surprise. I wanted to reuse the CQL and I couldn’t at that point use a Prepare because of a bug, since SimpleStatement has a nearly identical API I figured it’d be a one for one replacement so my friend and I started with:

var batchStmt = new BatchStmt();
var v2Insert = new SimpleStatement("insert into v1_readonly.documents " +
"(key, column1, value) values(?, ?, ?);");
batchStmt.Add(v2Insert.Bind("1", "DateStamp", "foo")) //1
.Add(v2Insert.Bind("1", "EprsVersion", "foo")) //2
.Add(v2Insert.Bind("1", "ExternalDocId", "foo")) //3
.Add(v2Insert.Bind("1", "RecordType", "foo")) //4
.Add(v2Insert.Bind("1", "Status", "foo")) //5
.Add(v2Insert.Bind("1", "XmlRecord", "foo")); //6
session.Execute(batchStmt);

and expected to get 6 rows but instead got the last record:

Cassandra - Batch loading without the Batch keyword

Thu, 28 Aug 2014 13:10:55 +0000

Cassandra - Batch loading without the Batch keyword

ATTENTION:

This post is intentionally simplistic to help explain tradeoffs that need to be made. If you are looking for some production level nuance go read this afterwords.

Batches in Cassandra are often mistaken as a performance optimization. They can be but only in rare cases. First we need to discuss the different types of batches:

Unlogged Batch

A good example of an unlogged batch follows and assumes a partition key of date. The following batch is effectively one insert because all inserts are sharing the same partition key. Assuming a partition key of date the above batch will only resolve to one write internally, no matter how many there are as long as they have the same date value. This is therefore the primary use case of an unlogged batch:

Cassandra & Rails: Cequel Batch Support

Tue, 15 Jul 2014 00:00:00 +0000

Cassandra & Rails: Cequel Batch Support

Cassandra On Rails using Cequel Atomic Batch Support When I first tried to use Cassandra with Rails over a year ago the lack of a good native (IE not thrift) driver and a good mapper was a show stopper for the project I was on. Today the situation has improved markedly with the cql-rb driver proving stable in usage, and the excellent Cequel ActiveModel capable library switching to use cql-rb.

I’d like to cover a common use case that’s not been handled by ORM style mapping libraries very well, atomic batches. Cequel has support for this even though you have to hunt a bit for it and it works well. Consider the following scenario, you’ve got 2 related tables, and you want one to update when the other one changes.

Buffer Cache Makes Slow Disks Seem Fast, Till You Need Them.

Sun, 20 Apr 2014 00:00:00 +0000

Buffer Cache Makes Slow Disks Seem Fast, Till You Need Them.

Linux has this wonderful thing called the buffer cache (for more detail read [here](http://www.tldp.org/LDP/sag/html/buffer-cache.html) ). In summary, it uses all your free ram as a cache for file access. Because of buffer cache you can easily get under 1 millisecond response times.

However, this sets a lot of people up for a trap. Imagine you buy a “database server” with a 5400RPM hard disk at Best Buy, while you’re there you pick up an extra 8 gigs of RAM. After loading the latest Ubuntu and restoring a 1 gig customer database backup. You check to see how much RAM you’re using and you have 2 gigs free. You test the new server out and records are coming off that server at an unbelievable speed, your happy, your boss is happy, you look like a genius.

Data Modeling for Scale with Cassandra

Thu, 17 Apr 2014 00:00:00 +0000

Data Modeling for Scale with Cassandra

## Reads and the perils of index tables.

I frequently see index tables in Cassandra being used to allow a One Source Of Truth. It’s important to remember when designing a truly distributed system relational algebra really doesn’t scale, and in memory joins will only get you so far (very little really). So as we often do in relational systems when we have an expensive dataset that is to expensive to calculate on the fly we create a materialized view. In Cassandra it’s helpful to think this way for all datasets that would require joins in a relational system.

How to become in polyglot in 5 hard steps.

Fri, 31 Jan 2014 00:00:00 +0000

How to become in polyglot in 5 hard steps.

With today’s world of programming languages where many languages are better at certain tasks than other’s you’ll find it useful to learn multiple languages over the course of your career (as well as keeping your skill sets current).

Here are some tips I’ve had to learn the hard way:

Step 1: Your second language should be similar to your first

Your brain will confuse a lot of language decisions with computational necessities. Warts from your first language will show more, and syntax will make your head explode.

An Evernote backed Journal using Vim/Emacs

Sun, 07 Apr 2013 00:00:00 +0000

An Evernote backed Journal using Vim/Emacs

I journal quite a bit and my holy grail has been using my favorite text editor (Vim or Vim bindings) with Evernote to store the everything in a smart searchable format. Today I stumbled onto a neat little tool that makes this all happen called Geeknote http://www.geeknote.me/. It’s written in Python and works fine on my Mac.

Installing Geeknote

In the directory of your choosing run the following script. This will checkout the latest copy from source and allow you to login:

#!/bin/sh

# Download the repository.

git clone git://github.com/VitaliyRodnenko/geeknote.git

cd geeknote

# Launch Geeknote and go through login procedure.

python geeknote.py login

#change vim to whatever you want it to be

python geeknote.py settings –editor vim

Then add the following script and execute it from whatever directory you want to install Geeknote into

#!/bin/sh

#change checkout_dir to match where you’ve checked out the latest

checkout_dir=~/Documents/geeknote

#change notebook to whatever notebook you use as your journal

notebook=Journal

title=$1

if [ -z “$1” ]

then

title=$(date +%Y-%m-%d)

fi

echo creating a note named $title in the $notebook notebook

python $checkout_dir/geeknote.py create –title $title –notebook $notebook –content “test”

python $checkout_dir/geeknote.py edit –note $title –notebook $notebook –content “WRITE”

Writing Journal Entries

My script is named journal so for me I just type either of the following:

journal #creates a note in journal with todays date as the title

journal custom_title #no spaces allowed and will use the title specified

Summary

With a few simple scripts and in moments you two can be writing notes in the command prompt. I highly recommend you extend these scripts to your needs or just use the Geeknote command prompt as you see fit.

Ruth's Story

Mon, 04 Feb 2013 06:28:45 +0000

"Ruth's Story"

Ruth Ann Svihla came into this world screaming and angry on October 30 2011 at 6:57 am. She was and has always been a beautiful, intelligent child that brought us a great deal of joy, but she was born with many challenges to overcome.

Cleft

The first such challenge we faced was a pretty severe cleft lip and pallet. Although this turned out to the be the most minimal of her problems, it was significant at the time. She was unable to eat on her own that first day, and we had to feed her through a nose tube. Attempts to teach her how to eat were met with frustration, both from my daughter and from us as well. I went to bed that night distraught and worried at challenges of being a new father of “special needs” child. Would my wife and I be able to cope?

Rail 3.1 CI setup with Jenkins, Test Unit & SimpleCov on OS X Lion.

Sun, 25 Sep 2011 12:04:54 +0000

Rail 3.1 CI setup with Jenkins, Test Unit & SimpleCov on OS X Lion.

I recently had to setup a build server for some rails work I'm doing. Still wanting to support my other projects I setup Jenkins. I ran into several issues.

Running Jenkins as a hidden user

First I noticed that jenkins was running as the “daemon” user, this obviously wasn’t going to work for github and rvm needs. So I did some googling and had some guides to get Jenkins running as a specific user. I did the following (sourced from http://colonelpanic.net/2011/06/jenkins-on-mac-os-x-git-w-ssh-public-key/ ).

The difficult definition of professional software development

Fri, 04 Feb 2011 22:51:00 +0000

The difficult definition of professional software development

Here are some of the contradictory phrases (and a few paraphrases) I’ve overheard used to define what is “good” and “bad” code.

Code should always be well commented
Maintainable code has unit tests and well named methods therefore needs little if any comments
Class explosion is to be avoided at all costs
Many small simple well named classes are the key to well organized code
Unit tests really don’t test anything useful
Dependency injection leads to an unusable mess
Dependency Injection is a very useful tool for making your code extensible and easily reused”
Inheritance leads to solid code reuse
Inheritance is the strongest coupling of your code you can have
Functional programming simplifies and minimizes the complexity of your code
Functional programing just makes my eyes bleed!
Static global references are ‘just programming’ and something you have to learn how to manage to make simple easy to understand code
Static global spiderwebs are crippling to maintenance and program lifetime

Java IoC containers and classpath scanning (or what I’ve been looking for from .NET for months)

Sun, 19 Sep 2010 19:01:00 +0000

Java IoC containers and classpath scanning (or what I’ve been looking for from .NET for months)

Frustrated with the typical way I saw IoC used in Java where every example I found involved thousands of lines of XML and/or Java code to configure Java beans or components. This is very different than IoC typically used in .NET where most IoC containers allow you to "autowire" in their terminology up every class in an assembly with a couple of lines of code. Having been coding in that fashion for several years in .NET I was dismayed when none of my fellow Java coders that I worked with or knew personally had any concept of the equivalent functionality, and instead informed me the IDE would be my help in maintaining these massive XML files.

Not content with their answers I burrowed into Spring, Guice and PicoContainer docs and found “classpath scanning” which is roughly
equivalent to “autowire” in .NET. Below is an example of this in Spring 3:

Anti-Pattern: Too much of your application is about interacting with external resources

Fri, 16 Jul 2010 02:24:00 +0000

Anti-Pattern: Too much of your application is about interacting with external resources

Firstly, what am I talking about? Applications that meet some of the following descriptions:

Stored procedures with a fair amount of conditional logic or complicated business rules buried in a sub-query (some would argue sprocs at all).
Web pages with lots of conditional output again encoding business rules in snippets of view logic. if else checks that display the latest price for something if before and certain date or a default price.
The majority of ‘unit tests’ require a database, or web and application server to be up and running.
The majority of code files make reference to language default I/O or database libraries.
Hours are spent trying to determine if differences in version of infrastructure are the cause of certain bugs.
Changing database schema results in hours of refactoring, as hundreds of querys and sprocs are hunted through to see if this index or that index is being properly hit.

Why is this an anti-pattern? First before I bore you with endless reasoning behind SOLID principles, proper OO etc if you’re reading this and you’re of a bent that disagrees with me so far entirely..none of that will mean anything to you, and none of it will mean anything to your customers so let me put this in the most practical terms I can.

Project Management in Java: A Confused .NET Developer’s Perspective

Mon, 12 Jul 2010 11:00:00 +0000

Project Management in Java: A Confused .NET Developer’s Perspective

When I was first introduced to workplace Java the amount of ways one could define a project appeared to be restrictive, confusing and a point of frequent friction. While those things may all be true, it’s a great deal better than I would have expected since I was coming from a world where an upgrade to a newer version of Visual Studio requires your whole team to sync up to that same version! So how do cross IDEA teams get any work done in the Java workplace? Do Java OSS projects require everyone uses Eclipse, NetBeans, or IntelliJ? Does every new IDE need to have import/export filters for all of the other existing ones and if it becomes popular do the already present IDE’s have to include this? Which system is the best and what matters?

To answer these questions let break down what makes a project and while doing so compare and contrast the Java perspective with the .NET one.

MySQL 5 Performance Tuning Toolkit

Fri, 02 Jul 2010 02:57:00 +0000

MySQL 5 Performance Tuning Toolkit

Recently we’d played with table partitioning and because of the limitations of it and some decisions we’d made a very long time ago we ended up spending a couple of days tracking down hotspots. In the process I picked up a few tricks, I’m not even an authority on any of this but I figured it’d help out others in the same spot.

Logging Toolkit

If you have a slow loading page or test and have no idea which one of the thirty or so queries going on is the cause how to do you find it. Slow query log and logging queries that are not using indexes. Adding the following to your my.ini, my.cnf or whatever your mysql config file is named on your system and you’ll be able to get a list of offenders to further investigate.

Java Dependency Management with Apache Ivy

Tue, 29 Jun 2010 13:10:00 +0000

Java Dependency Management with Apache Ivy

Not wanting to ditch your already built well working ant scripts for the plugin-centric view of Maven, especially if your project structure doesn’t line up quite right with Maven’s point of view? Enter Apache Ivy which like Maven can automatically download and resolve all of your dependencies and their dependencies for you with just a few simple lines of XML (if only I could get rid of the XML Part).

Pros

Hibernate Connection Pooling: why isn’t the default one for production?

Sat, 26 Jun 2010 13:50:00 +0000

Hibernate Connection Pooling: why isn’t the default one for production?

Hibernate unlike NHibernate comes with a variety of connection pooling options. The three primary ones of which I’m aware are [Proxool](http://proxool.sourceforge.net/), [Apache DBCP](http://commons.apache.org/dbcp/), and [c3p0](http://sourceforge.net/projects/c3p0/) . I myself have only so far used c3p0 and it works quite well having saved me from a couple of jams so far where the default one was in use previously. First why not use the default one? Because the official Hibernate documentation says in a big yellow box marked caution the following words:

Dynamic DNS with Amazon EC2 Linux and EveryDNS

Sun, 06 Jun 2010 18:02:32 +0000

Dynamic DNS with Amazon EC2 Linux and EveryDNS

So I finally sat down and did the math and found out Amazon EC2 was quite a bit cheaper than what I’d been paying for hosting as long as I was willing to prepay for at least a year. However, with EC2 you are getting a dynamic ip’s so what to do? Well I’ve been using EveryDNS for years for my dns hosting and despite it’s recent acquisition by Dyn, Inc it’s still works the same. The scripts that are described below can be downloaded here, what follows is a brief walkthrough.

Download EveryDns update script from http://www.everydns.com/dynamic.php and place it in something like /usr/local/bin.
Create a convenience script in /usr/local/bin that will get your external ip and have it contain the following lines and call it external-ip.sh . This is using Amazon’s suggested curl command to find out your EC2 instance Ip address.
Then create a script for the actual dynamic DNS update in /usr/local/bin/ and call it updatedns-sh. This is using the set command to grab the output of external ip and storing it in the variable $1, and then passing that value to the update command. NOTE: this will update the dns record of ALL your domains with everydns set to dynamic. If you need to use specific domain names there is a –d flag for the eDNS client.
create a cron job on your linux distro of choice but the cron job line should look like the following.

For those wishing to comment I’ve cross posted to my blogger account.

Projects in Java with Maven 2

Sun, 23 May 2010 00:00:00 +0000

Projects in Java with Maven 2

NOTE: due to issues with spam I’ve turned off comments, I’ve cross posted on my [old blogger account](http://ryansvihla.blogspot.com/2010/05/projects-in-java-with-maven-2.html) if you have comments.

For those of you who don’t know Maven is a build tool/dependency manager/project model. Those in the Microsoft space can probably imagine MSBuild + the ability to download all dll’s for you.

What I liked

Dependency Resolution. Automatic downloading of dependencies rocks in concept. Just specify library name and version and when you compile again it’s there. No need to check in jar’s into your source tree.
IDE Independence. In practical terms lets you use whatever IDE you want with no import/export. Intellij, Eclipse and Netbeans all understand maven as a full project format. So your team can all have different IDE’s and not create havoc with one another.
Good Default Project Structure. Tests are in the same location by default, resources for tests and your prod code are in expected places.

What I hated

Inefficient with already downloaded dependencies. It will check remote repositories every time you build even when you specified version number. Now I can see why they did that but not as a default. I mean sure version 1.12 may have had a bad bug and the project hotfixed in a new one with the same version number, but I’d say that’s really unlikely. In a larger project with several repositories the difference in time to build between this and an alternate dependency manager Apache Ivy is stark.
Very opinionated view of the build process. Those of you used to (N)Ant or Rake will miss the lack of control. The AntRun plugin will help mitigate some of the lose of control.
HOME/.m2/settings.xml . Specific system wide settings do not belong in a build language. The bad heavy friction ideas that it enabled were legion. Just one example was I had to connect to my work VPN even when working on my home OSS projects hosted on github.
Way too XML for for sometimes simple things. To specify the version of java target language version see the figure below, this specifies Java 1.6.

Hudson CI Server: setting up remote slaves AND restrictive security together

Wed, 14 Apr 2010 00:00:00 +0000

Hudson CI Server: setting up remote slaves AND restrictive security together

### NOTE: this applies to Hudson version 1.352

Ok so yesterday I was setting up a publicly accessible build server, that was not to be viewable to anonymous sources. So I configured Hudson to “matrix security” and disallowed all access to anonymous see fig 1-1.However, once I did this my slaves stopped working entirely. Now I could have allowed some anonymous access but here was the long sought for trick.

Come see my Boo talk at Alamo Coders on the 14th

Tue, 13 Apr 2010 00:00:00 +0000

Come see my Boo talk at Alamo Coders on the 14th

UPDATE: added links for video, slides and source

S****ubject: Boo: The Extensible Little Language On The CLR You’ve Never Heard Of.

When: Wednesday April 14th at 5:45pm

Where: New Horizons In San Antonio map

Summary:

I will be covering Boo basics, advanced features, and building a small domain specific language.

Resources:

slides and source code are at https://github.com/rssvihla/Presentations/tree/master/boo-dsl-talk/ .sorry keynote for now when final touches are made will export to PDF.

Domain Specific Languages with Boo: AST Macros

Mon, 05 Apr 2010 00:00:00 +0000

Domain Specific Languages with Boo: AST Macros

For those of you who don’t know what Boo is its a statically typed CLR language with Python like syntax that lets you extend it’s compiler, and the language itself easily by giving you access to the AST (Abstract Syntax Tree) and compiler’s context directly. This gives you very powerful tools for building your own language or Domain Specific Language or DSL from here on out. Some examples of DSL’s include rSpec and Fluent NHibernate. In fact the entire subject of what is a DSL and what types of DSL there are and how to create a proper DSL could be a book itself and a fascinating one at that.

Which is why I’ve been reading Ayende’s book DSLs in Boo: Domain Specific Languages in .NET. To make sure I understood the concepts I’ve taken to building a toy BDD DSL called bSpec, it’s got a long way to go to be something useful and I may not care to take it that far, however I did get my brain wrapped around a really cool thing called AST Macros.

Newbie Win32 Development Using Python

Sat, 16 Jan 2010 00:00:00 +0000

Newbie Win32 Development Using Python

First a disclaimer. I’m not in anyway shape or form an experienced win32 systems programmer. I’ve always done application development or systems administration scripting. That in-between area where you get out your C/C++ compiler and start dealing with pointers is completely alien to me. This has created problems in the past when I need to do something outside of the scope of C# and found myself staring at MSDN docs in C trying to extrapolate the equivalent C# code, and typically being disgusted at having to use hand rolled Structs with calls to Marshall.GetLastWin32Error().

Worse still I end up with a nasty implementation that requires slow, fragile integration tests to verify behavior or very verbose mirror interfaces where I’m testing order of calls. Look at the following code sample to read a reparse point’s target directory from code project:

Adventures in Meta programming in Python

Fri, 08 Jan 2010 05:03:00 +0000

Adventures in Meta programming in Python

With my recent work in Python win32 programming I’ve had a real need for AAA style mocking framework. Unable to find anything that I’ve been completely happy with I started my own simple mocking framework and got to learn some Python style meta programming in the process. I’ve found out a lot about the depth of the Python 2.x object model over the last 2 weeks and here are some of the nicer things I’ve found (updated as per Chris Taveres below):

function_class = MyClass.foo.im_class

This is useful if you pass in an instance method somewhere and you want to be able to access its attached class. Decorators on instance methods normally use this, but I needed it to get a nicer syntax on my asserts, I wanted this:

Take 3: Python, ISP, IoC, and OCP need a fundamental rethink.

Fri, 20 Nov 2009 00:00:00 +0000

Take 3: Python, ISP, IoC, and OCP need a fundamental rethink.

In response to Julian’s thoughtful Dynamic Languages and SOLID Principles I’d have to argue he is about 95% there but is missing the last critical links needed to view this in a whole different light. For ISP, Julian says it all:

Ultimately, I don’t think ISP is changed by Python, it’s just kind of irrelevant, for better or worse.

Exactly! Why even bother mentioning in the context of dynamic languages. A developer that understands it fully and one that does not will code the same way. Whereas LSP is still taught in dynamic languages, I’ve never seen it called that but I’ve seen many admonishments about subclass consistency, and not making client code aware of the differences, and is something that requires mentioning. With ISP we could ignore it’s entire existence and developers would code the same.

Take 2: Why we use SOLID in static languages and how we get the same functionality for cheap in dynamic languages

Thu, 19 Nov 2009 00:00:00 +0000

Take 2: Why we use SOLID in static languages and how we get the same functionality for cheap in dynamic languages

One of the things we do pretty well at Los Techies is explaining SOLID principles and why they make our code more maintainable and if you’re not familiar with our work on SOLID, read Chad Meyer’s post to get an understanding on the Los Techies perspective. Now that you’ve taken a bit of time to read that the following chart takes SOLID’s benefits and then shows how they apply to dynamic languages:

I recant my IoC! IoC containers in dynamic languages are silly.

Mon, 16 Nov 2009 00:00:00 +0000

I recant my IoC! IoC containers in dynamic languages are silly.

After a year or so of solid Alt Dot Net infection (as far as infections go its a pretty awesome one to have), I decided to give Python a try again for more than one off sysadmin tasks, and to actually dive into it as a newly minted “Agilista”.

However, I had a problem..there were no non-painful IoC containers in Python (sorry to the other authors of IoC frameworks in Python like Spring Python and Snake Guice, I know you try and I respect the effort). Ultimately, I could not imagine coding anymore without something to handle all my registration for me, that’d dynamically inject in my dependencies, give me hooks for contextual resolution, and give me rich interception support.

Alt.Net Evangelism And What We Could Do Better

Mon, 14 Sep 2009 00:00:00 +0000

Alt.Net Evangelism And What We Could Do Better

If we assume our goal as the Alt.Net community is to improve software development as a whole, we’re doing a pretty bad job. Not that many people have even heard of the concepts we espouse, and those that have but didn’t convert become hostile to our ideas.

My theory is because we focus so much on things that at the surface seem extraordinarily complex. Our terminology while correct, is acryonym laden and hard to grasp the real meaning. We ourselves sometimes lose site of why we even care about some of those principles, why we enacted them in the first place and what pain they caused to begin with.

Playing With Haskell

Wed, 02 Sep 2009 00:00:00 +0000

Playing With Haskell

My tech book club is starting to look at Haskell. I'm as big of a language addict as anyone so I wanted to dive in early and this is the overview of what I've found so far

Statically Typed But Dynamically Inferred

for those of you familiar with Boo this is a familiar concept to you. Once a type is made that type will not autoconvert. For Example:

Simple BDD/TDD

Wed, 02 Sep 2009 00:00:00 +0000

Simple BDD/TDD

Todays theory is **most tests and specs should be very short (2-3 lines), have at most a setup for context establishment, avoid the majority of test framework features as they should be used as an exception and not as a rule.** Note: I practice BDD nowadays and do not like using the term “test”, but these rules apply for BDD as well as TDD, and I imagine most of you practice TDD so I’ll be using primarily TDD terminology.

If you look at my first attempts at TDD they are very similar to my current BDD work with only changes in language. But the contexts, conditions and tests are similar. Most importantly they’re easy to read. They’re all very short, very descriptive, make limited use of arguments in the NUnit Attributes, and make very limited use of mocking.

Dynamic Loading of .Net Assemblies

Mon, 31 Aug 2009 00:00:00 +0000

Dynamic Loading of .Net Assemblies

Problem: You have dependencies which can be in a number of directories outside of the directory your app is running in. DotNet AppDomains are heavy to say the least and very high in ceremony. You do not want to deal with secondary app domain creation.

Solution: Override AssemblyResolve event on your current app domain.

     public class AppDomainLoader : IAppDomainLoader
     {
        
         private ResolveEventHandler _searchAssemblyMethod = _searchDir;
         public ResolveEventHandler FindAssemblyMethod
         {
             set { _searchAssemblyMethod = value; }
         }
         private static string[] _directoriesToSearch;
         private  static Assembly _searchDir(Object o, ResolveEventArgs e)
         {
             foreach (var s in _directoriesToSearch)
             {
                 var filetoload = s +  e.Name.Split(‘,’)[0] + “.dll”; //bad one liner that parses the first part of the assembly name and adds .dll at the end
                 if (File.Exists(filetoload)) //sees if the file is even in the directory
                 {
                    return Assembly.LoadFrom(filetoload);  //returns assembly once found, a bit naive and does not handle version conflicts well.
                 }
             }
             return null;
           
         }
      
         public void Search(string[] directoriesToSearch)
         {
             _directoriesToSearch = directoriesToSearch; //sets directories to search to allow entry point for dynamic searching
             AppDomain.CurrentDomain.AssemblyResolve -= _searchAssemblyMethod; //cleans up previous events, may not be needed.
             AppDomain.CurrentDomain.AssemblyResolve += _searchAssemblyMethod; //registered the event
 
         }
     }
 

Now whenever there is a missing Assembly, the _searchAssemblyMethod will be fired, and all directories given to the Search method will attempt to find an assembly with the matching name.

Focused and Narrow; Superficial and Broad

Sun, 30 Aug 2009 00:00:00 +0000

Focused and Narrow; Superficial and Broad

In my 20’s I got exposed to a large variety of work situations and was able to succeed in a variety of technical fields (admin, developer, hardware tech, etc). I had the time back then to not have to choose which direction I went with things, I went broad but deep into each field I was interested in. Every couple of years I’d go head long into another field and read dozen books, setup labs experiment like crazy, use whatever connections I’d made to get my foot in the door somewhere and then make sure I delivered in big ways, and most importantly keep reviewing what I’d done in the past to make sure it actually did work and did work well.

This obsessive learning habit had a great side effect, as long as I stuck to slightly related fields I learned exponentially more. Learning about networking made helping customers with a variety of issues far easier when I was just a lowly computer tech. When I was trying to automate customer installations in a repeatable way, learning software development even at a trivial level made things possible that were not otherwise. I got to see how everythign interconnected.

Hero Mode Refactoring” AKA Enemy Of Brown Field Development

Fri, 14 Aug 2009 00:00:00 +0000

“Hero Mode Refactoring” AKA Enemy Of Brown Field Development

What do I mean by “Hero Mode Refactoring”? We’ve all worked with code that wasn’t easily made testable. Most of the time we see a big ball of mud and have no idea where to begin. Sometimes however we have an epiphany, a \_great\_idea_ , a “cold fusion” moment (that ends up being poignant in so many ways).

Filled with the excitement of our vision we dive head long into our work, replacing concrete classes with interfaces, removing big swaths of redundant code, and in general making changes you should only make with a an already unit tested code base. Before we know it, 5 things are broken in weird ways, and it now takes us days of manual verification to figure out what works and what doesn’t. That is hero mode refactoring at its best.

Introducing SpecMaker “Rspec style” BDD in C#

Sat, 08 Aug 2009 00:00:00 +0000

Introducing SpecMaker “Rspec style” BDD in C#

So I’m certain this will be met with mixed response, because really .Net already has several decent BDD frameworks and many of you will chastise me for adding yet another framework when really BDD has nothing to do with what testing framework you use. So why you ask?

Most of the BDD frameworks I’ve looked at are Acceptance style and trying to make stories into executable code (NBehave, StoryTeller, Fitnesse.Net ,Acceptance, etc). I want something that describes to other developers in a behavior centric way what my code is doing (like RSpec’s default DSL does). This is not aimed at business analysts.
The other RSpec style framework for C# I’ve played with while very nice, did not go over well with people I’ve tried to introduce to BDD.
Using NUnit in a context BDD style is normally what I do but produces a lot of artifacts, is underscore heaven and provides no guidance to newer practitioners of BDD.

With all this in mind how does SpecMaker improve our situation at all? First lets look at how I might approach a BDD test with NUnit

Windsor style convention over configuration in StructureMap

Fri, 07 Aug 2009 00:00:00 +0000

Windsor style convention over configuration in StructureMap

So I was trying to get my BDD project SpecMaker working on Mono only to find out my usual IoC container of choice Castle Windsor does not play nice. StructureMap I’ve heard plays nice in Mono, and I know a bunch of people that use it, so I gave it a whirl.

My original Windsor configuration looked something like this:

  var kernel = new DefaultKernel();
  kernel.Register(Component.For().ImplementedBy(factory)); //load implementation
  kernel.Register(
                 AllTypes.Pick().
                 FromAssembly(GetType().Assembly).
                 WithService.FirstInterface().  //regardless of naming first interface is listed as service
                 Configure(c=>c.LifeStyle.Transient));
 

I changed this to the what I thought was the StructureMap equivalent:

Discussion: Mentoring, Knowledge Transfer and Different Styles Of Learning

Fri, 17 Jul 2009 00:00:00 +0000

Discussion: Mentoring, Knowledge Transfer and Different Styles Of Learning

I’ve been very fortunate that for most of my adult life, even before I got into tech I’ve been “the boss”, and had gobs of mentoring opportunities. I enjoy it and more importantly I know I will have a greater effect for my company’s growth through teaching than I would by solely focusing on my advancement.

My students have been visual learners, “doing” learners, audio learners, those that learn by rote memorization with photographic memories, “theorists” that are fascinated by the theory of things far more than the practice, and those that can’t memorize anything unless they see an entire system in place. Each type or combination needs to have information presented in different ways.

Lessons From My First Attempt At Bringing Agile Into A Non-Agile shop.

Sun, 12 Jul 2009 00:00:00 +0000

Lessons From My First Attempt At Bringing Agile Into A Non-Agile shop.

Since I’m moving on from my current employer I figure now is a great time to share my post-mortem of my time there, specifically what worked, what didn’t and what did I learn.

Iterations Of Any Length

We tried 3 weeks, 2 weeks, 1 week, monthly and eventually just sort of tossed the system out all together as our team got shrunk from 5 to 2. We did find get a fairly desirable result ultimately but more on that later. Ultimately our problem was especially as we got smaller, dealing with our “particular” realities made iterations tend to fall apart, we were spending a fair amount of time trying to torture ever more atomic stories together. Cutting iterations out all together and focusing on features being completed was MUCH more time effective for us.

VAN: Castle Monorail, Active Record and Brail talk at Wednesday 8PM CST (-5 GMT), by yours truly.

Tue, 07 Jul 2009 00:00:00 +0000

VAN: Castle Monorail, Active Record and Brail talk at Wednesday 8PM CST (-5 GMT), by yours truly.

Last week I did a talk on Windsor and it went..ok. I’m not used to online presentations and am more experienced in front of crowds that I can interact with.

So between last week’s experience and some advice from Agile Joe this talk I’ll be aiming for more of a screen cast feel. The talk should go for close to an hour and will be open for questions afterword.

Python Web Framework Series – Pylons: Part 7 Refactoring Deployment and Wrap up

Sat, 04 Jul 2009 00:00:00 +0000

Python Web Framework Series – Pylons: Part 7 Refactoring Deployment and Wrap up

Lets take a look at our existing site and what we can do to clean it up and add some badly needed functionality, however that is beyond the scope of my series. So I’m going to leave some hints for the remaining functionality to get you started.

Refactoring Ideas

Replace default authkit login with custom login. HINT: make a sign-in link on your public page. then redirect to a custom login page that uses authkit. Trying it any other way will make you lose hair.
Refactor database access into a class or two. This will make the db access more testable and allow you to monkey patch in different behavior later. HINT: If your controllers can avoid having to know anything about SQLAlchemy you’ve done your job.
Replace return “rsvihla” in the users.py module with actual authkit user. HINT: just access pylons request object: “request.environ.get(“REMOTE_USER”)”
Create admin ui for users, with roles. HINT: look at Pylons Book for Authkit ideas here.
Split threads into categories and then give the ability to respond to posts. HINT: just more controllers and views.
Makes parts of your view visible depending on if a user has certain rights or not. HINT: while this is not what you’ll find in the Pylons Book I highly suggest not have gobs of if/else checks in your views. Make small functions that you can unit test easily which will change what is visible depending on role, then have your views call these. See what I did with my user.py module for a concept.

Deployment

This almost completely depends on your situation with hosting. If you are using something like WebFaction you just have to upload files and follow their directions. I found it trivial to have a production site up and running with them.

Python Web Framework Series – Pylons: Part 6 Basic Authorization With AuthKit

Fri, 03 Jul 2009 00:00:00 +0000

Python Web Framework Series – Pylons: Part 6 Basic Authorization With AuthKit

Last post we left off with very basic database access, and testing story completed. Now we’re going to look at basic Authorization and Authentication with AuthKit. NOTE: most of this post is just an aggregation of a couple of chapters in the [Pylons Book](http://pylonsbook.com/en/1.0/simplesite-tutorial-part-3.html) since this setup is a good base starting point. Read the previous link to the Pylons Book for more in depth coverage of this topic.

Setting Up AuthKit

First lets make sure we have AuthKit installed: easy_install AuthKit. For this post we’re working with AuthKit 0.4.3, your mileage may vary if you read this post in the future and are using a different version. Now that we have Authkit installed open up pylonsforumconfigmiddleware.py add the following imports:

Python Web Framework Series – Pylons: Part 5 Testing Models

Thu, 02 Jul 2009 20:09:00 +0000

"Python Web Framework Series – Pylons: Part 5 Testing Models"

UPDATE: was an error caught by Govind (who is turning out to be my unofficial proofreader). I’ve made a correction in the thread mapping for “dateadded” property in the _previous_ article. If this you are caught up with and _error indicating there is no dateadded property on thread_ make sure to edit your **model\_\_init\_\_.py** file to match the previous article and rebuild your db so that everything is happy. Please bear with me as I hone my tutorial writing skill set.

When we last left of with our Pylons forum we had a had just successfully created a post, and then could immediately retrieve that same post. However, we kind of skimped on the testing story so lets fill in the gaps and do some refactoring as a bonus.

Python Web Framework Series – Pylons: Part 4 Introduction For Database Support With SQL Alchemy

Mon, 29 Jun 2009 02:54:00 +0000

"Python Web Framework Series – Pylons: Part 4 Introduction For Database Support With SQL Alchemy"

We last left off with Views with Mako, now Pylons does not enforce on you an ORM at all, so you can use hand crafted SQL if you prefer. However, since I’ve done enough of that for a career or two we’re going to use my Python ORM of choice and the preferred one for Pylons SQLAlchemy.

Where does SQLAlchemy fit in as an ORM?

If you used NHibernate, you should feel pretty close to right at home with SQLAlchemy. If you come from an ActiveRecord, Entity Framework, Subsonic, or even Linq2Sql background this will be a bit more hand management than you are used to. If that is the case, I suggest Elixir (which I’ve heard great things about) or SQLObject (which I’ve used as well and works fine).

Python Web Framework Series – Pylons: Part 3 Views with Mako

Thu, 25 Jun 2009 04:53:00 +0000

"Python Web Framework Series – Pylons: Part 3 Views with Mako"

This is a huge post and I should have split this into several smaller ones so please bear with me while I get my series format tweaked. We last left off with Controllers, Views and Testing with a basic test, basic view and basic controller. Now with our basic scaffold built we can focus on making our views reusable, dynamic, and more pleasant to look at.

First things first

First before we dive too deeply into Mako, lets setup a default home page by doing the following:

VAN: Doing 2 Part Series on Castle development July 1st and 8th at 8pm

Thu, 25 Jun 2009 00:00:00 +0000

VAN: Doing 2 Part Series on Castle development July 1st and 8th at 8pm

I know this isn't related to my Pylons series and worry not there is no interruption in that, but I’m giving a couple of talks on Virtual Alt.Net. The first one I've done before and you may have heard it at Alamo Coders, the second one is all new and it'll give me opportunity to clean up and release my lightweight plugin framework for monorail.

For those of you who don’t know Virtual Alt Net is held over Live Meeting here

Python Web Framework Series – Pylons: Part 2 Controllers, Views and Testing

Wed, 24 Jun 2009 13:22:00 +0000

"Python Web Framework Series – Pylons: Part 2 Controllers, Views and Testing"

We last left off with Getting Started and having created our “pylonsforum” project and generated our first controller.

Source for home.py

import logging 

from pylons import request, response, session, tmpl_context as c 
 from pylons.controllers.util import abort, redirect_to 

from pylonsforum.lib.base import BaseController, render 

 log = logging.getLogger(__name__) 

class HomeController(BaseController): 

     def index(self): 
         # Return a rendered template 
         #return render(‘/home.mako’) 
         # or, return a response 
         return ‘Hello World 

Source for test_home.py

Python Web Framework Series – Pylons: Part 1 Getting Started

Tue, 23 Jun 2009 14:00:00 +0000

Python Web Framework Series – Pylons: Part 1 Getting Started

This article assumes you have Python 2.6 and Setuptools already installed on your machine and that you’re install SQL Alchemy 0.54 and Pylons 0.97

Overview

Pylons is a component based MVC web framework. It, like a lot of more recent MVC frameworks, is borrowing some ideas and concepts from rails in a less “opinionated “ way.

Out of the box Pylons has a preference for using SQL Alchemy for ORM (similar to NHibernate in philosophy) and Mako template engine (uses Python code for markup) . Now if you prefer SQLObject , or the Elixer dialect of SQL Alchemy for ORM and Genshi template engine pylons supports them all (and others). More importantly this means you have choice down the road to match your personal preferences or particular project complexity needs.

Python Web Framework Series

Mon, 22 Jun 2009 13:31:00 +0000

Python Web Framework Series

I've wanted to dive more in depth into Python's web frameworks for some time and get to know them the way I know Castle Monorail.

What better way to learn something in depth than to have to know it well enough to show to someone else.Therefore, I plan on posting a relatively in depth walkthrough of Pylons (and if the interest is there Django and Turbogears as well).

Engineering or Customer Service which is more important?

Sun, 07 Jun 2009 19:16:00 +0000

Engineering or Customer Service which is more important?

I've been reading a lot about lean/toyota way lately, mixing that with my past experience as an IT Consultant and throwing that up against a bit of an Ivory tower I fell into with XP that left me unable to deal with the following scenario:

You’re reviewing code either yours or someone elses right before release. you find the code strewn with anti-patterns and unnecessary conditionals. You are due to ship tomorrow. The code works but is not maintainable, do you ship?

Castle ActiveRecord and registering classes at runtime

Wed, 03 Jun 2009 06:02:00 +0000

Castle ActiveRecord and registering classes at runtime

*EDIT: this is an old post copied over from my [old blog on Los Techies](https://lostechies.com/ryansvihla/2009/06/03/castle-activerecord-and-registering-classes-at-runtime/), I have kept it here for archival reasons.*

I use the following trick for adding ActiveRecord classes after the fact for our in-house plug-in architecture. Thanks to whichever blogger/mailing list I picked this up from so long ago.

Scenario 1: Class(es) to load inherit from same base class as current running ActiveRecord classes

var holder = ActiveRecordMediator.GetSessionFactoryHolder();
//register your subclasses here
ActiveRecordStarter.RegisterTypes(typeof(Employee), typeof(Customer));

holder.RegisterSessionFactory(
 holder.GetConfiguration(typeof(ActiveRecordBase)).BuildSessionFactory(), typeof(ActiveRecordBase)
);
//reloads session factory without this types will not be registered. 
//I believe this is actually a "feature" in Castle trunk. may be fixed now.

This is all you need to register extra classes that are accessing the same database as the application is using.

IronPython how cool it is

Thu, 28 May 2009 05:08:00 +0000

IronPython how cool it is

*EDIT: this is an old post copied over from my [old blog on Los Techies](https://lostechies.com/ryansvihla/2009/05/28/ironpython-how-cool-it-is/), I have kept it here for archival reasons.*

Briefly, IronPython is an implementation of Python in the .Net runtime. This allows you access to .Net framework goodness while programming in a dynamic language. The current stable version 2.0.1 maps to CPython 2.5.This allows me to do fun fun things like use my python project to access a c# project. follows is a python script using Pinsor and accessing a .net dll.

Context and Spec style testing and my approach to BDD

Fri, 22 May 2009 00:21:00 +0000

Context and Spec style testing and my approach to BDD

*EDIT: this is an old post copied over from my [old blog on Los Techies](https://lostechies.com/ryansvihla/2009/05/22/context-spec-style-testing-and-my-approach-to-bdd/), I have kept it here for archival reasons.*

I borrow heavily my approach to testing from a combination of Ayende’s Rhino Tools tests, and my reading of the Rspec beta book. But I think I’ve stumbled onto something I’m happy with and I can generate reports out of. Let’s go over some basic rules first:

Glad to be here

Thu, 21 May 2009 22:02:14 +0000

Glad to be here

EDIT: this is an old post copied over from my old blog on Los Techies, I have kept it here for archival reasons. The contact info and links below are almost certainly no longer accurate

Hi, my name is Ryan Svihla, and I’m the newest member of Los Techies; and I’d like to provide some background for most of you wondering, “who is this guy.”

I’ve recently moved to San Antonio, TX, and love it here. I don’t miss the winters and humidity of the farm states where I previously lived. I greatly appreciate the local developer community and learning at the feet of people like Jasdeep, Jason, Joe, Josh](https://lostechies.com/blogs/joshua_lockwood/default.aspx), Anne, Phil, and many others.